About Time
“It’s a poor sort of memory that only works backwards,” the Queen remarked.
—Lewis Carroll, Through the Looking Glass
One important aspect of episodic memory is that it locates events in time. Although we are often not clear precisely when remembered events happened, we usually have at least a rough idea, and this is sufficient to give rise to the general understanding of time itself. Episodic memory allows us to travel back in time, and consciously relive previous experiences. Thomas Suddendorf called this mental time travel, and made the important suggestion that mental time travel allows us to imagine future events as well as remember past ones.1 It also adds to the recursive possibilities; I might remember, for example, that yesterday I had plans to go to the beach tomorrow. The true significance of episodic memory, then, is that it provides a vocabulary from which to construct future events, and so fine-tune our lives. Natural selection cannot operate on the basis of past recollection per se, but only on what it contributes to survival in the present and future.
There is growing support for this suggestion. What has been termed episodic future thinking,2 or the ability to imagine future events, emerges in children at around the same time as episodic memory itself, between the ages of three and four.3 Patients with amnesia are as unable to answer simple questions about past events as they are to say what might happen in the future.4 Clive Wearing, the amnesic musician described in the previous chapter, constantly complains that he has no past, and is stuck in the present. He is forever under the impression that he has just woken up. His plight is captured in the title of his wife’s book, Forever Today. Amnesia for specific events, then, is at least in part a loss of the awareness of time.
Figure 9. Midsagittal fMRI scans of brain activation elicited by imagining past events (left) and future events (right). Activated areas are shown as white. (Reprinted from Addis et al. 2007, with permission from Elsevier.)
Brain-imaging studies confirm the close connection between episodic memory and imagining the future. Indeed the idea of mental time travel was anticipated in some of the earliest studies of cerebral blood flow by the Swedish physiologist David Ingvar, who concluded that activation in the prefrontal cortex was especially important in providing the internal connection between past and future. He wrote:
On the basis of previous experiences, represented in memories, the brain—one’s mind—is automatically busy with extrapolation of future events and, as it appears, constructing alternative hypothetical behavior patterns in order to be ready for what may happen.5
Using more refined techniques based on functional magnetic resonance imaging (fMRI), Daniel L. Schacter and his colleagues at Harvard University have shown that parts of the medial temporal lobe, as well as the prefrontal cortex, are activated when people are prompted to remember past events or to imagine future ones. Schacter and colleagues refer to this as a core network.6 But of course people can usually distinguish remembered past events from imagined future ones, and some studies have shown in several brain areas greater activation in response to future than to past events, perhaps indicating that imagining the future requires more intensive imaginative construction.7 Some studies, though, have shown greater activation in recalling past events than in imagining future ones,8 so the evidence is inconsistent. Reliving the past also involves active construction,9 sometimes to the detriment of veracity, and I suspect that people may occasionally even confuse imagined future events with those that actually happened. The brain itself seems to hardly know the difference.
Is Mental Time Travel Unique to Humans?
He said “What’s time? Now is for dogs and apes! Man has Forever!”
—from A Grammarian’s Funeral by Robert Browning
Like Browning’s grammarian, Suddendorf and I also claimed that mental time travel, including episodic memory, is uniquely human.10 The same idea was also suggested by the German psychologist Wolfgang Köhler. Köhler was working at a primate research facility maintained by the Prussian Academy of Sciences in the Canary Islands when World War I broke out. He was marooned, and occupied his time studying the behavior of nine chimpanzees contained in a large outdoor pen. His work is famous for showing that chimpanzees sometimes solve mechanical problems through the use of insight rather than mere trial and error.11 Köhler nevertheless concluded that, for all their improvisational skills, chimpanzees had little conception of past or future. Chimpanzees are our closest nonhuman relatives, so Köhler’s observation suggests that mental time travel evolved at some point after the split between the hominin and ape lineages.
The idea that mental time travel is uniquely human has nevertheless been challenged. The most serious challenge comes, not from apes, but from birds. Some show prodigious memory for the locations in which they have hidden food for later consumption. Clark’s nutcrackers seem to be among the most prolific, storing seeds in thousands of locations and recovering them with high (but not perfect) accuracy.12 This need not mean, though, that they remember actually caching the seeds—they may simply know where the seeds are. It has been suggested that the case for episodic memory would be strengthened if the birds could be shown to have some memory for when the seeds were cached, implying memory for time as well as place. Memory that includes storage of what, where and when, dubbed www memory,13 has been taken by some researchers as sufficient evidence for episodic memory, and therefore of mental time travel, at least into the past.
The challenge to demonstrate www memory in birds has been taken up by Nicola Clayton and her colleagues at Cambridge University, using scrub jays. In clever experiments, Clayton allowed the birds to cache food in different compartments of ice-cream trays.14 They were given two different foods to cache, worms and peanuts. They prefer worms to peanuts, and went to the location where worms had been cached rather than to where the peanuts were hidden, but only if the worms had been recently cached. If too much time had elapsed, and the worms were likely to have decayed and become inedible, they went instead to where the peanuts had been cached. This suggests that they remembered not only where they had hidden a specific food, but when they had hidden it. This implies further that they might have been traveling mentally back in time to the point at which they had originally cached the food.
These birds are also apt to steal each other’s cached food. If a bird that had itself stolen food was watched while caching its own food, it later privately recached it, presumably in order to prevent the watcher from stealing it. It takes a thief to know a thief. Recaching also depended on which bird was watching. Clayton and her colleagues have found that recaching was more likely to occur if the predatory watcher was a dominant bird than if it was a subordinate bird.15 Thus we might add a further w—the birds seem to know what, where, when, and who. Their behavior also seems to suggest mental time travel into the future as well as into the past, since recaching might be taken to imply anticipation of a future theft.
Meadow voles, it has been claimed, can also remember what, where, and when.16 In an experimental study, male voles were first allowed to explore two chambers, one containing a pregnant female 24 hours prepartum, and the other containing a female that was neither lactating nor pregnant. Twenty-four hours later, they were again given access to the chambers, which were now empty and clean, and spent more time exploring the chamber that had contained the previously pregnant female than the one that had housed the other female. This suggests that they had remembered the pregnant female, and understood that she would now be in postpartum estrus, a state of heightened receptivity. In another condition they first explored a chamber containing a female in postpartum estrus and another containing a female that was neither lactating nor pregnant, and was not in estrus. Twenty-four hours later they were again allowed to explore the cages, now clean and empty, and showed no preference for the chamber that had housed the female in estrus. This suggests that they realized the female would no longer be in a state of heightened receptivity.
Nevertheless it is perhaps too early to conclude that these animals—birds or voles—can really travel mentally in time in the way that we humans do. The retrieval of worms or nuts depending on how long they have been cached need not imply that the jays actually remember the act of caching, as distinct from knowing when and where the food was cached. I know that Captain James Cook reached New Zealand in 1769, but I do not remember it; indeed I wasn’t even born at the time, despite what my students may think. I don’t even remember being told it. In the case of the birds, a simple time tag attached to the memory, as a kind of “useby date,” could be sufficient. Similarly the voles’ understanding of when a female is likely to be receptive could be based on knowledge of ovulation cycles, with a variable time marker attached to the memory. A distinction can be drawn between knowing when something happened and knowing how long ago it happened, and a recent study shows that rats, in choosing where to find food in a maze, are governed by the latter, not the former.17 Rats don’t travel mentally in time, and there is still no strong reason to suppose that scrub jays or voles do either.
And do the scrub jays recache their food when a likely thief is watching because they actually envisage a future act of theft? Not necessarily. Their behavior might simply be learned through the association of a watcher and subsequent loss of food.
But we are in difficult territory here, since psychological concepts are slippery, and it is often all too easy to explain a set of results in different ways to suit one’s theoretical predilections. Even if birds or voles can be said to have a degree of mental time travel, the context in which it operates is narrow. Scrub jays are specialists in the caching and recovery of food, and their behavior in this context has evolved subtleties that may not extend to more general activities. And all animals are specialists in mating. Somehow I doubt that they can lie back and reminisce about the past or dream about the future, as we humans do.
One theory that affords a potential test of mental time travel into the future is the so-called Bischof-Köhler hypothesis, which states that only humans can flexibly anticipate their own future mental states of need and act in the present to satisfy them.18 We go to the supermarket to buy food even though not hungry at the time, or attend expensive and often dull university courses in the interests of a later lucrative career. There have been several attempts to refute the hypothesis that such behaviors are unique to our species. In one study, for example, scrub jays, given the choice of two foods to cache, ceased to cache one of those foods if it would be available later when they were hungry.19 However, this behavior could have been based on learning rather than on the anticipation of which food would be available later, when hunger struck20—there seems no good reason to suppose that the birds traveled mentally in time to a future occasion when they would be hungry. In all cases so far described, processes simpler than mental time travel can account for the findings.
The manufacture of tools for specific purposes is also sometimes taken as evidence for mental time travel into the future. Here, I’m afraid, we must yet again defer to birds, since New Caledonian crows are able to manufacture tools from twigs and bits of wire to solve mechanical problems, although chimpanzees make tools of at least comparable complexity.21 Of course many examples of tool use may be a simple matter of improvisation to solve immediate problems, rather than planning for a more distant future. More convincing evidence of future thinking comes from the crows’ ability to shape leaves from pandanus trees. These leaves have spikes along one side, and are therefore useful for inserting into holes containing grubs, which are then caught on the spikes and extracted. The crows shape these tools in tapered fashion, so that the wider end is held in the beak and the narrow end inserted into the hole. Given the ubiquity of these tools, the standardization of design, and the evidence for cultural transmission of technique,22 it is highly unlikely that they are simply the outcome of improvisation. These tools are planned.
Chimpanzees also show many cultural variations, in the way in which they use tools as well as in other behaviors.23 And some groups of chimpanzees do store hammers and anvils for years of use in cracking nuts, although this need not mean that they actually imagine themselves using these tools in the future.24 Nevertheless there has been a recent claim that orangutans and bonobos, at least, save tools not needed in the present for use up to 14 hours later, which might suggest mental time travel,25 although it is not entirely clear that the animals were not responding simply on the basis of past associations, rather than actively imagining a future event.26 The bonobo Kanzi is said to be able to lead someone to a location where he knows something to be located, but again this need not imply that he remembers the act of visiting that location previously. The critical distinction between knowing and remembering is actually very difficult to test in nonhuman species.
In humans, at least, mental time travel implies the conscious acting out of episodes, whether past or future, which further suggests recursion. That is, a conscious episode is embedded in present consciousness. This can proceed to deeper levels, as when I remember that yesterday I planned an episode—perhaps a dinner party, for some date in the future. It may be this recursively embedded structure that differentiates our own time-governed behavior from that of other species.
Time and the Human Condition
We might concede that there is limited ability for remembering past events in other species, and perhaps even for imagining future ones, but this pales beside the extraordinary impact that mental time travel has had on the human condition. Manufactured stone tools go back some two million years in hominin evolution, and probably reflect an emerging understanding of the concept of time. Events located in time prey heavily on our conscious lives, whether in reminiscing, regretting past mistakes and hoping to eradicate them in the future, planning dinner parties and weddings, imagining what life will be like after retirement. We are ruled by clocks, calendars, diaries, appointments, anniversaries—and taxes. Indeed our concept of time now extends back to the very origin of the universe, which came about, we are told, in a Big Bang somewhere around 13.7 billion years ago.27 This claim of course leads to metaphysical anxiety about what was going on before that.
Figure 10. Descartes discovers he exists through time (author’s drawing).
All species are driven to some extent by time, as in seasonal changes or the daily cycle of the sun, but these depend on bodily rhythms as well as on external signals, such as temperature or ambient light. Humans have measured time in deliberate fashion, to guide intentional behaviors. We measure it in seconds, minutes, hours, days of the week, months, years, centuries, millennia, epochs, eras, eons. These can be understood recursively, in that each element of time is embedded in a larger one—like the smaller fleas on the backs of larger ones in Augustus de Morgan’s poem. Time can also be understood as cyclic rather than linear.28 Seconds within minutes, minutes within hours, hours within days, days within weeks, weeks within months, months within years, and so forth, all circle around in a dazzling array of wheels within wheels. Explicit labels, such as clock times, specific days of the week, or months of the year, can help us locate specific events in time, independently of bodily rhythms or seasonal fluctuations.
As I suggested in chapter 1, it is perhaps through iteration rather than true recursion that we understand the nature of time. Just as tomorrows extend indefinitely into the future, so yesterdays extend indefinitely back into the past. This generalized notion of time comes with costs, such as a lingering sense of guilt for remembered indiscretions, or anxiety over anticipated events that might repeat earlier unpleasant ones. The most fateful consequence of mental time travel, though, may be the understanding that we will all die. The lines of David Watts’s hymn “Oh God our Help in Ages Past,” based on Psalm 90, tell the truth we all must face:
Time like an ever rolling stream,
Bears all its sons away,
They fly forgotten as a dream
Dies at the opening day.
The understanding of death, along with theory of mind, is of course an unpalatable aspect of the human condition, but it is moderated to some extent by the understanding that time continues beyond death. Our lives, perhaps, can be continued in those of our offspring. One function of religions, though, might be to instill the notion that life itself can continue after death, whether in the form of Heaven, Hell, Valhalla, or Nirvana. This belief in life everlasting may offer comfort through the promise of rewards after death, thereby providing some consolation from the sheer bleakness that death otherwise suggests. It can also be used to manipulate behavior, as in threats of hellfire and damnation if we do not conform to certain rules of behavior, or heavenly rewards if we do. Acts of terrorism through suicide, as in the destruction of the Twin Towers in New York on 11 September 2001, or Japanese kamikaze pilots who died for their emperor in World War II, might be considered extreme examples of such manipulation, although one might argue that all soldiers who submit to warfare are similarly manipulated, and celebrations of the glorious dead continue long after a war is over. The offer of life after death, with associated rewards and punishments, is remarkably ingenious, since there seems no way in which we can be either gratified or disappointed—at least if these emotions are restricted to the living. This may explain the origin of faith.
The relation of religion to the sense of time is born out by the Pirahã, the small Amazonian community that we encountered in chapter 2. It was the Pirahã’s lack of a sense of time, and consequent failure to understand what religion is all about, that led Daniel Everett to forego his career as a missionary among them and become an atheist—and a professor.29
The Pirahã notwithstanding, we can trace something of the prehistory of mental time travel through burial sites. There is some evidence that the Neandertals buried their dead, although the reasons for burial may have been more practical than religious. In some early human burials, though, symbolic material is added to the grave, suggesting that those who buried their dead had some notion of a spiritual life that continued after death of the body. At least from the evidence to date, such burials were restricted to our own species, Homo sapiens.30 Perhaps the earliest example is from a burial site at Qafzeh Cave in Israel, in which the head of a deer was placed on the body of a child who was buried. This burial is dated at about 100,000 years ago, and is associated with early anatomically modern humans.31
The understanding of time is of course not all bad news. Sometimes the knowledge that time will pass is a consolation, as when Viola in Shakespeare’s Twelfth Night, masquerading as a man, finds herself in an impossible situation, and is moved to say
O Time, thou must untangle this, not I;
It is too hard a knot for me t’untie!
And it is arguably the understanding of time that has shaped human destiny. Instinctive behaviors like dam-building in beavers or migrations in birds are future-oriented, but in limited and inflexible ways, at least compared to the extraordinary way in which we humans can plan our futures, or manufacture buildings, transportation devices, household conveniences, suitable clothing, and impressive CVs, that help us guarantee future prospects.
According to Ayurveda, the ancient Indian science of life, “You are what you eat.” Much more obviously, though, we are what we remember, and what we plan. Mental time travel, then, provides us with the concept of a conscious self that extends in time. The psychologists Hazel Markus and Paula Nurius wrote of “possible selves,” derived from representations of the self in the past and including representations of the self in the future.32 The pioneering psychologist William James expressed much the same idea when he wrote of “potential social Me” as distinct from “immediate present Me” and “Me of the past.”33 The concept of different possible selves provides the primary motivation that guides our future endeavors. As Markus and Nurius put it, “I am now a psychologist, but I could be a restaurant owner, a marathon runner, a journalist, or the parent of a handicapped child.”34 The motivation can operate both positively and negatively; I can imagine myself as a roaring success, whether at parties, on the rugby field, or in scientific achievement, or I can see myself as a dismal failure.
Mental Time Travel and Fiction
The concept of mental time travel helps explain the fragility of episodic memory, documented in the previous chapter. The importance of episodic memory, then, lies not in providing a detailed record of the past, which it does very poorly, but rather in its role in constructing future scenarios. As we have seen, episodic memory itself is essentially a construction; Ulric Neisser recently wrote, “Remembering is not like playing back a tape or looking at a picture; it is more like telling a story.”35 It is a process whereby we establish our own identities, often in defiance of the facts.
This leads to fiction itself. The same constructive process that allows us to reconstruct the past and construct possible futures also allows us to invent stories. We humans are addicted to folktales, legends, novels, movies, plays, soap operas, and everyday gossip. It is the power of recursion that makes these things possible. Critical to all of them is language, the device that enables us to share our memories, future plans, and dreams. More of that, though, in the next chapter.