|
|
Memory does not comprise a single unitary system, but rather an array of interacting systems, each capable of encoding or registering information, sorting it, and making it available by retrieval. Without this capability for information storage, we could not perceive adequately, learn from our past, understand the present, or plan for the future.
A. Baddeley
1999
Human memory is intricate and diverse. Over a lifetime, you will learn vast amounts of information and retain it for various amounts of time. The important role played by memory in virtually all aspects of human life is evident when one considers the severe consequences of the memory deficits characteristic of Alzheimer’s disease (Ryan, Rossor, & Fox, 2015). This disease is characterized in its late stages by its victims becoming lost in familiar environments and failing to recognize immediate family members. Memory is involved not only in identification and recognition of places and people, but also in remembering task goals and maintaining a “set,” or appropriate readiness, to perform particular tasks. It is also involved in maintaining information in a readily available form for comprehending new information, solving problems, and retrieving facts and procedures that one has learned in the past.
It is an unappreciated fact that memories can be distorted and that a person’s ability to retrieve his memories depends on many environmental and contextual conditions (Marsh & Roediger, 2013). This characteristic of memory can lead to many types of memory failures and errors. The human factors professional needs to know and appreciate that how well an operator learns and remembers plays an important role in his or her ability to perform within a human–machine system. In most circumstances, successful performance of the system depends on the operator’s ability to recognize and retrieve information from memory. Human factors specialists can improve human performance by ensuring that environments and training materials will support learning, retention, and retrieval of important information.
There are a lot of ways that we can talk about memory, including the kind of information that is “stored” and the cognitive processes that allow storage and retrieval of information. One prevalent way of thinking about the kinds of information that can be stored in memory is to distinguish between semantic and episodic memory (Tulving, 1999): Semantic memory refers to a person’s basic knowledge, such as the fact that a dog is a friendly animal with four legs that barks, whereas episodic memory refers to specific events (or episodes), such as that Fido bit the mail carrier this morning. Different processes might require us to talk about specific features of the environment that promote or interfere with the storage and retrieval of different kinds of information.
In the first part of the chapter, we focus primarily on episodic memory. A very popular way of thinking about episodic memory is called the modal model (Atkinson & Shiffrin, 1968; Thorn & Page, 2009; see Figure 10.1). In this model, when information is first presented, it is retained with almost perfect fidelity for no longer than a few seconds in the form of a sensory memory. Only some of this information is then encoded in a more durable form, called short-term memory. Short-term memories are retained for a period of around 10–20 s unless they are kept active through rehearsal, or covert repetition. Finally, some of the information in short-term memory is transferred to long-term memory and retained for an indefinite duration.
FIGURE 10.1The three-store memory model.
The present chapter organizes our knowledge of human memory around the distinction between sensory, short-term, and long-term memories. Each type of memory has distinct properties that affect human performance in a wide variety of situations. We will describe these properties, as well as the important factors that affect the acquisition, retention, and retrieval of information of each. Although we know that the modal model of memory is not a completely accurate portrayal of the human memory system, it is useful for organizing our knowledge of human memory. In the last part of the chapter, we examine the role of memory in the comprehension and retention of written and spoken information.
The sensory effects of stimuli persist for a short period of time after they have been removed from the environment. For example, when a letter is displayed briefly, its perceived duration exceeds its physical duration (e.g., Haber & Standing, 1970); that is, the display visibly persists. Researchers have shown that it is possible to retrieve information from the persisting representation of the display, in addition to the display itself. These and related findings have been taken as evidence for the existence of sensory memories that are thought to exist for each sensory modality.
Research on visual sensory memory was inspired by a memory limitation called the span of apprehension, known since the 1800s (Cattell, 1886). This span refers to the number of simultaneous, briefly displayed visual stimuli that can be recalled without error. For example, an array of letters may be presented to you briefly, and your task is to report as many of the letters as possible. This is a whole report task. If the array is small enough (four or five letters), then you can report all of the letters correctly. However, when the arrays are larger, you will report only a subset, usually around four or five letters—the same number of letters in a small display that you could report correctly. This is the span of apprehension.
Although large arrays of stimuli cannot be identified with complete accuracy, observers often claim that they can see the whole display at first but it “disappears” before all of the stimuli can be identified (Gill & Dallenbach, 1926). Ingenious experiments by Sperling (1960) and Averbach and Coriell (1961) established that observers could indeed see more letters than they could report. Instead of whole report, these researchers used a procedure known as partial report, in which only some of the letters are to be reported. Sperling showed people three rows of four letters very briefly. At varying times after the display, he then played a high-, medium-, or low-frequency tone as a cue to indicate whether they should report the top, middle, or bottom row. When the tone occurred immediately at the offset of the array, the cued row could be reported with almost perfect accuracy, regardless of which row was cued. Because people could not know in advance which row would be cued, this suggests that they could see all of the letters in the display. With whole report it appeared as though they could see only four or five letters, but with partial report it is apparent that they could see all of them. This difference is called the partial report superiority effect.
If the tone was delayed by as little as one-third of a second, report accuracy decreased to the four or five letters measured by the span of apprehension (see Figure 10.2). That is, report accuracy was at the level that would be expected if only four or five items were available. For delay times between the end of the letter array and one-third of a second later, partial report superiority could be seen to fade away. When a distracting array of random contours immediately followed the display, it interfered with sensory memory of the display, resulting in no partial report superiority even when the cue was not delayed.
FIGURE 10.2Partial report accuracy as a function of delay.
These and other results obtained by Sperling (1960) led to the conclusion that visual stimuli persist visibly in a high-capacity sensory-memory store that decays within a second and is susceptible to disruption by subsequent visual stimulation. Sperling’s research was extremely influential, resulting in many experiments exploring the properties of what came to be called iconic memory for visual stimuli and sensory memory more generally (see Cowan, 2008; Nairne & Neath, 2013). These experiments showed that informational persistence of the type evidenced by partial report superiority is distinct from visible persistence of the type demonstrated by tasks that require temporal integration of visual information (Coltheart, 1980; 2009; see Chapter 5). Completion of these tasks cannot be done unless the stimulus is “visible.” For example, Haber and Standing (1970) asked people to estimate how long a briefly flashed array of letters was visible. They perceived that the array lasted longer than its actual duration. In another experiment, Eriksen and Collins (1967) asked people to report a three-letter nonsense syllable that could only be identified after integrating two successively presented random-dot patterns. They could accomplish this easily when the interval between the two patterns was less than 50 ms, but their report accuracy decreased drastically to about chance levels with an interval of a third of a second.
Although alternative measures of visible persistence correlate highly with each other, they do not correlate much with the information persistence measure of partial report performance (Loftus & Irwin, 1998). For example, the estimated duration of visible persistence is less than that of information persistence. Also, whereas visible persistence decreases as stimulus duration and luminance increase, partial report accuracy increases. Thus, it is generally accepted that the visible persistence that you can “see” is different from the informational persistence that results in the partial report superiority effect.
TACTILE AND AUDITORY SENSORY MEMORIES
Sensory stores with properties similar to those of iconic memory seem to exist for the other senses. In particular, sensory stores for touch and audition have been examined (Bliss, Crane, Mansfield, & Townsend, 1966; Darwin, Turvey, & Crowder, 1972). In the same way that visual sensory memory can be disrupted by a distracting visual array, auditory sensory memory can be disrupted by a distracting auditory stimulus (e.g., Beaman & Morton, 2000).
The human factors specialist needs to remember that memory for auditory information can be disrupted by distracting auditory stimuli. This point is illustrated in a study by Schilling and Weaver (1983). Telephone operators working directory assistance at a local utility were told to say “Have a nice day” at the end of each transaction. Schilling and Weaver wondered whether this parting message could interfere with callers’ memory for the telephone numbers. Subjects in their experiments were instructed to obtain and dial numbers under situations similar to those for real directory assistance clients. On each trial, the subject dialed 411, requested and received a prerecorded seven-digit phone number, and then attempted to dial the number. In one condition, the phrase “Have a nice day” immediately followed the prerecorded number, whereas in other conditions the subject heard either a tone or nothing after the phone number. Fewer phone numbers were dialed correctly in the “Have a nice day” condition than in the other two conditions. In the “Have a nice day” condition, subjects had the most difficulty remembering the last two digits of the number, the ones most likely to still be in auditory sensory memory. Thus, the telephone company’s attempt to be polite may have actually interfered with the client’s goal of remembering a phone number.
WHAT IS THE ROLE OF SENSORY MEMORY?
Our early understanding of sensory memory was that it served as temporary storage for sensory information to get ready for further processing. For instance, it was suggested that visual sensory memory creates a continuous perception of the world by integrating discrete visual images across saccadic eye movements (e.g., Breitmeyer, Kropfl, & Julesz, 1982). However, the duration of visible persistence is too short to serve this purpose, although it still sometimes may be important in integrating temporally separate events (Loftus & Irwin, 1998). Creating a continuous perception is more likely a role played by auditory sensory memory, because integration across short time periods is necessary for comprehension of complex stimuli such as speech and music (Crowder & Surprenant, 2000).
Another possibility is that persistence is simply a consequence of imperfect temporal resolution within the sensory systems and nothing more (Loftus & Irwin, 1998). No matter what the role of sensory memory in human information processing, the important point for the human factors specialist is that the effects of sensory stimulation will persist for a brief period of time after the stimulation is removed. These effects may serve as a substitute for the physical stimuli (Rensink, 2014) and might influence an operator’s judgments about what she is perceiving.
You have probably at one time or another used a website—or even a telephone book!—to look up a phone number to call someone. After finding the number, you probably repeated the number to yourself until you dialed the number. If you were distracted, you probably forgot the number and had to look it up again. Experiences like these suggest that there is a short-term memory of limited capacity. Information in short-term memory must be “rehearsed” to be retained.
Short-term memory limits the performance of operators in a wide variety of situations. For example, an air-traffic controller must remember such things as the locations and headings of many different aircraft and the instructions given to each one (Garland, Stein, & Muller, 1999). Similarly, radio dispatchers for a taxi company must remember the taxis that are available and their locations. For tasks that rely on short-term memory, performance can be affected greatly by how information is presented and how the task is structured.
Two studies that revealed a lot about how short-term memory works were conducted in different laboratories by Peterson and Peterson (1959) and Brown (1958). These researchers showed people three consonants (e.g., BZX) to remember on a trial. This triplet can be recalled easily not just seconds later but minutes later if the rememberer is not distracted. However, the people in this experiment were required to count backward by threes from a three-digit number until they were instructed to recall the letters. Brown and Peterson and Peterson assumed that the mental activity required to count backward would prevent rehearsal of the letters, causing them to be lost from memory. After 8 s of distraction, only about half of the letters could be recalled correctly, and after 18 s, very few could be recalled (see Figure 10.3). This suggests that without rehearsal, short-term memory for the letters lasted only a few seconds.
FIGURE 10.3Recall performance as a function of a filled retention interval in a short-term memory task.
The rapid forgetting that occurs when rehearsal is prevented reflects two types of memory errors (Estes, 1972). A transposition or order error occurs when the correct items are recalled but in the wrong order (e.g., BZX could be recalled as BXZ). An intrusion or item error occurs when an item that was not in the list is recalled (e.g., BZX could be recalled as BGX). These two kinds of errors seem to be due to different kinds of processes (Nairne & Kelley, 2004). Order errors tend to occur more frequently than intrusion errors when the items must be remembered for only a short period of time. As time increases, item errors increase. This suggests that memory for the order in which items occurred is lost more quickly than memory for the items themselves. So, if an operator’s task does not require remembering the exact order in which information occurred, task performance will not suffer as much from delays in responding.
An important constraint on the role of rehearsal was demonstrated by Keppel and Underwood (1962). They showed that recall is virtually perfect for a single set of items even after 18 s. However, after the third or fourth set of items is presented, recall deteriorates to pure guessing. So, short-term forgetting does not occur by “decay” alone. It seems as though sets of items presented at different times can interfere with each other. Proactive interference refers to memory for earlier presented items interfering with memory for later items. Proactive interference can be reduced, improving short-term memory, if items are presented only once every few minutes (Peterson & Gentile, 1963). Proactive interference also can be reduced by changing the semantic characteristics of later items from those of previous items (for example, changing word categories from fruits to flowers) or, to a lesser extent, by changing physical characteristics (for example, changing font type, size, or color; Wickens, 1972). In sum, the accuracy of short-term memory can be improved by increasing the intervals between successive messages and by making each message somehow distinctive.
The kind of information that is stored in short-term memory seems to have a strong acoustic component. That is, although semantic or visual information may be stored (Shulman, 1970; Tversky, 1969), much of the information is represented by how it sounds (Conrad, 1964). Conrad showed that intrusion errors were acoustically similar to the original items. For example, in an experiment similar to those of Brown (1958) and Peterson and Peterson (1959), the visual letter B was often remembered as the acoustically similar letter V, even though the two letters look nothing alike. This means that sets of items that are acoustically confusable (that is, sound alike) will produce more short-term retention errors than sets that are not.
You may already be familiar with a classic paper entitled The Magical Number Seven, Plus or Minus Two: Some Limits on Our Capacity for Processing Information, by George Miller (1956; see also Chapter 4). In this paper, Miller measured the capacity of short-term memory as seven plus or minus two chunks, or units, of information. If you try to remember isolated digits or letters, this capacity represents the number of them that you can recall correctly. However, if you group the items into larger chunks, recall can be vastly improved. For example, if a person attempts to remember the list CBSABCNBC as a string of separate letters, the string is nine chunks long, greater than the seven chunks that can be held easily in short-term memory. Because short-term memory is overloaded, recall of all nine letters may be difficult. However, if the person recognizes that the string can be coded as acronyms of three major television networks, CBS, ABC, and NBC, the string can be encoded as three chunks and will therefore be much easier to remember. More recent research confirms that short-term memory capacity is a function of chunks rather than the objective number of items to be remembered (Cowan, Chen, & Rouder, 2004).
Strings of digits are often used for such things as telephone numbers, bank accounts, customer identification, and so on. From a customer’s perspective, these numbers are essentially random. It is very difficult to remember random strings of digits, so chunking is an important strategy that can be used to remember them. Some important chunking strategies involve the size of the chunk and the modality in which the information is presented. Wickelgren (1964) showed that lists of digits are easiest to remember if they are organized into groups of a maximum of four. Grouping provides a better benefit when digits are presented auditorily rather than visually, because people tend to chunk visual digits into pairs even when they are not grouped (Nordby, Raanaas, & Magnussen, 2002).
IMPROVING SHORT-TERM RETENTION
The limited capacity of short-term memory has implications for any situation that requires an operator to encode and retain information accurately for brief periods of time. Memory performance can be improved by using techniques that minimize activities intervening between presentation of the information and action on it, using sets of stimuli that are not acoustically confusable, increasing the interval between successive messages, making the to-be-remembered material distinct from preceding material, and grouping the information into chunks.
Several of these techniques were exploited in a study by Loftus, Dark, and Williams (1979) that examined communication errors between ground control and student pilots in a short-term memory task. Memory was tested for two types of messages: (1) a place for the pilot to contact plus a radio frequency (for example, “contact Seattle center on 1.829”) and (2) a transponder code (for example, “squawk 4273,” which means set the transponder code to 4273). The codes were presented in two-digit chunks (for example, “forty-two, seventy-three”) or as unchunked single digits (for example, “four, two, seven, three”). In a low-memory-load condition only one of the two message types was presented, whereas in a high-load condition both were presented. After the message(s), the pilot had to read off a sequence of rapidly presented letters for varying amounts of time. When this task was completed, the pilot wrote down the original message(s) on a piece of paper.
Recall was worse in the high-load condition, which required more information to be retained. Moreover, recall of the radio frequency was better when the transponder code was chunked, suggesting that chunking made more short-term memory capacity available for other information. Loftus et al. (1979) concluded that as little information as possible should be conveyed to a pilot at one time. They also proposed that a delay of at least 10 s should intervene between successive messages, because they observed that on any trial, the longer it had been since the immediately preceding message had been presented, the more memory performance improved. Additionally, they showed that the response to a message should be made as quickly as possible to avoid error, and that alphanumeric strings should be chunked whenever possible.
Not only does the size of the chunks influence the accuracy of short-term memory, but so does the nature of the chunks, whether numbers or digits. Preczewski and Fisher (1990) examined the format of call signs used by the military in secured radio communications. The U.S. Army uses two-syllable codes of the sequence letter-digit-letter (LDL) followed by the sequence digit-digit (DD). These codes make radio communication very difficult, and they change at least once a day. Preczewski and Fisher compared the memorability of the current code format (LDL-DD) with that of three other formats: DD-LDL, DD-LLL, and LL-LLL. Operators were presented with a call sign, which they were to recall later. During a 10-s delay between presentation and recall, the operators read aloud strings of letters and digits. Performance was best when one syllable was composed only of digits and the other of letters (DD-LLL) and worst on the current code. Thus, mixing letters and digits within chunks seems to be harmful, whereas mixing them between chunks is beneficial.
Specific alphanumeric characters differ in their memorability. We have already seen that confusions can be reduced by using letters that do not sound similar to other letters (Conrad, 1964). Chapanis and Moulden (1990) investigated the memorability of individual digits, as well as of doublets and triplets, within eight-digit numbers. People viewed a number for 5 s, then immediately entered it on a numeric keyboard. Many errors were made, as would be expected, because the length of the number was greater than the normal memory span of seven. The digit that was remembered best was 0. The remaining digits, in order of their memorability, were 1, 7, 8, 2, 6, 5, 3, 9, and finally 4. Doublets were generally easier to recall if they contained a zero or if they contained the same digit twice. A similar pattern of results was found for triplets. Based on these findings, the authors provide tables that designers can use to construct numeric codes that are easy to remember.
For many tasks, accurate performance requires not only that information be retained in short-term memory, but also that the information be acted on quickly. The time required for search and retrieval from short-term memory has been investigated extensively using a memory search task (Sternberg, 1966, 2016). In this task, which we introduced briefly in Chapter 4, observers are presented a set of one or more items (such as digits, letters, or words) to be held in short-term memory. Shortly thereafter, a single target item is presented. The observer is to indicate as quickly as possible whether or not the target was in the memorized set, usually by making one of two key presses indicating its presence or absence.
In Sternberg’s (1966) study, the memory set was composed of one to six digits, followed by a single target digit. Reaction time increased as a linear function of the memory-set size (see Figure 10.4). The rate of increase was approximately 38 ms per item in the memory set. Sternberg interpreted these data as support for the idea that a rapid, serial scan was performed on the memory set. That is, when presented with the target, the observer compared the target with each item in the memory set, one at a time. If each comparison takes 38 ms, for example, the function relating response time to memory-set size will be linear with a slope of 38 ms and with an intercept equal to the time taken by all other processes not involved in the comparison, such as perceptual and response processes.
FIGURE 10.4Memory search time as a function of set size and response.
Although Sternberg’s findings are consistent with those expected from a serial search of the memory set (see Sternberg, 2016), we can devise more complicated, non-serial processes (using, for example, variable comparison times) that will also predict linear reaction-time functions (Atkinson, Holmgren, & Juola, 1969; Townsend, 1974). Thus, Sternberg’s findings are not conclusive evidence for serial comparison processes. Moreover, other experiments have shown findings, such as faster responses to repeated or the last items in the memory set (Baddeley & Ecob, 1973; Corballis, Kirby, & Miller, 1972), that are inconsistent with the basic assumptions of the serial search model.
Regardless of the exact nature of the search process, the slope of the function relating reaction time to memory-set size is an indicator of short-term memory capacity. Items such as digits, for which the memory span is large, show a slope that is considerably less than that for items such as nonsense syllables, for which the span is smaller (Cavanagh, 1972). The slope can be used as an indicator of the demands placed on short-term memory capacity. Moreover, this measure of memory capacity can be isolated from perceptual and motor factors that affect only the intercept. Consequently, the memory search task has been used as a secondary-task measure to assess mental workload (see Chapter 9).
The memory search task was used by Wickens, Hyman, Dellinger, Taylor, and Meador (1986) to study the mental workload demands imposed on instrument-rated pilots at various phases of flight. An instrument-rated pilot is one who is qualified to fly on instrument readings alone, without visual contact outside the cockpit window. Wickens et al. asked the pilots to fly an instrument-only holding pattern and instrument-only landing approach in a flight simulator while performing a memory search task. The intercepts of the search functions were greater during the approach phase than during the holding phase, but the slopes of the functions did not differ. Consequently, Wickens et al. (1986) concluded that the approach phase increases perceptual and response loads. Because there was no change in the slope of the search function, they concluded that short-term memory load was no greater during approach than during holding. They recommended that pilots should not be asked to perform tasks that require perceptual-motor processing while landing a plane.
MODELS OF SHORT-TERM, OR WORKING, MEMORY
Recent work on short-term memory has focused on its function, which seems to be primarily one of temporarily storing and manipulating information. To distinguish this approach from the study of short-term memory, the term working memory is often used. Working memory is involved in performing calculations necessary for mental arithmetic, comprehending the meaning of a sentence, elaborating the meaning of material, and so on. As Jonides, Lacey, and Nee (2005) indicate, “Without working memory, people would not be able to reason, solve problems, speak, and understand language” (p. 2).
Baddeley and Hitch’s Working Memory Model
The most popular model of working memory yet developed is the one proposed by Baddeley and Hitch (1974), illustrated in Figure 10.5. The model has two storage systems, called the phonological loop and visuospatial sketchpad, and a control system, called the central executive. The phonological loop consists of a phonological store, in which the information is represented by phonological codes, and an articulatory rehearsal process that essentially involves saying the items over and over to yourself. Consistently with the findings from studies discussed earlier, memory traces in the phonological store are lost within a few seconds unless maintained by the articulatory rehearsal process. Evidence for this aspect of the model comes from a finding called the word length effect. In the word length effect, the number of words you can hold in short-term memory decreases as the number of syllables in those words increases. This might occur because it takes longer to rehearse words that have more syllables in them. The phonological loop might play a role in vocabulary acquisition, learning to read, and language comprehension.
FIGURE 10.5Working-memory model.
The visuospatial sketchpad is a store for visuospatial information. Similarly to the phonological loop, it is limited to a capacity of only a few objects (Marois & Ivanoff, 2005). The sketchpad is presumed to be involved not only in the memory for visually presented objects but also in visual imagery. The primary role of the sketchpad is to hold and manipulate visuospatial representations, which are important for artistic and scientific creativity. The central executive is an attentional control system that supervises and coordinates the phonological loop and the visuospatial sketchpad. This emphasis on attentional control suggests that working memory is closely related to attention. In fact, some of the tasks the central executive performs include focusing and dividing attention, switching attention from one task to another, and coordinating working memory with long-term memory. Baddeley (2000, 2003) later proposed a fourth component to working memory, the episodic buffer (see Figure 10.6). This subsystem integrates information from the other working memory subsystems and long-term memory into a common code. The central executive controls this subsystem, as it does the others, and the information in it plays a role in the formation of conscious experience.
FIGURE 10.6The revised working memory model.
This way of thinking about working memory implies that tasks should not interfere if they use different subsystems. For example, when people are asked to remember something (for example, a set of digits) to be recalled after another task is performed, the memory load (for example, the number of digits) often has little effect on the performance of that task. Baddeley (1986) varied the number of digits that people were to remember while they performed a reasoning or learning task that required central executive processes but not the phonological loop. Consistently with this theory of working memory, people were able to maintain a memory load of up to eight items without its interfering much with the performance of either task.
Another prediction from the model is that tasks sharing the same subsystem should interfere. Brooks (1968) reported evidence of such interference in the visuospatial sketchpad. Observers were asked to imagine a block letter, for example, the letter F (see Figure 10.7). They were told to mentally trace around the letter, starting from a designated corner and proceeding around the perimeter, responding to each successive corner with yes if it was either at the top or bottom of the figure, or no if it was in the middle. One group of observers responded vocally, and a second group tapped either of their index fingers. A third group had to point to a column on a sheet of paper that contained either Ys or Ns in a sequence of staggered pairs (see Figure 10.7). It took much longer to respond in the third group, presumably because the task required visually perceiving the locations of the Ys and Ns while at the same time visualizing the locations on the letter. These results are not due to the pointing responses being more difficult than the other kinds of responses, because pointing did not cause interference for a similar task that did not require the visuospatial sketchpad.
FIGURE 10.7Block-letter stimulus and “yes”-“no” response display.
In considering why the working memory model has been popular for many years, one reason given by Baddeley (2016, p. 121) is “Because it has just four components, each of which is relatively easy to understand, it can be applied to practical issues …” Human factors specialists and designers need to be aware that different tasks may interfere with each other to the extent that they require common components in working memory. According to Fiore, Cuevas, and Salas (2003, p. 511),
An understanding of the relation between [working memory] and complex task performance is essential for the development of effective training and system design. Because many of today’s tasks require one to monitor multiple system parameters, each potentially composed of input from differing modalities, and often require the integration of this information, we maintain that these differing task components uniquely impact systems of [working memory].
Another influential memory model is that of Cowan (1997), illustrated in Figure 10.8. Cowan’s model places even more emphasis than the working memory model on the relationship between attention and memory. In this model, the contents of the short-term store are activated long-term memories, and what we are consciously aware of at any moment (the objects or events that are in the focus of attention) is only a subset of the available information in the short-term store. Cowan’s model includes a brief sensory store, which corresponds to the stimulus traces that produce sensory persistence. The other component of sensory memory, informational persistence, is part of the short-term store. Like the working memory model, Cowan’s activation model includes a central executive that directs attention and controls voluntary processing.
FIGURE 10.8Cowan’s (1997) activation model of short-term memory.
The nature of visual imagery has been the subject of many working memory experiments. We already reviewed in Chapter 4 some studies that showed how observers can mentally rotate objects into similar orientations to determine whether the objects are the same. Other researchers, most notably Kosslyn (1975; Kosslyn & Thompson, 2003), argue that imagery is very much like perception. Several of their studies suggested that images are mentally scanned in a manner similar to visually scanning a picture. For example, Kosslyn, Ball, and Reiser (1978) had observers memorize a map of a fictional island containing a number of objects (see Figure 10.9). The experimenter then read aloud the name of one of the objects on the island. The observer was to imagine the entire map but focus on the specific object. Five seconds later, the experimenter named a second object. The observer was then to mentally scan to the location of the second object and press a response key as soon as it was reached. The farther apart the two objects were, the longer it took to mentally scan from the first to the second object.
FIGURE 10.9Map of a fictional island Kosslyn et al. (1978).
Kosslyn (1975) also provided evidence that it is more difficult to make judgments about components of small images than of large images. He asked observers to imagine a particular animal next to either a fly or an elephant. He argued that the animal would have a larger mental image next to the fly than next to the elephant. When the observer was asked to verify whether a certain property, such as fur, was part of the imagined animal, responses were faster if the animal was imagined as large (next to the fly) than if it was imagined as small (next to the elephant).
The concept of an imagery component to working memory has been extended to the notion of a mental model: a dynamic representation or simulation of the world (see Chapter 11; Johnson-Laird, 1983, 1989). Johnson-Laird has argued that mental models are a form of representation in working memory that provides the basis for many aspects of comprehension and reasoning (Johnson-Laird, Khemlani, & Goodwin, 2015). The key element of the mental model concept is that thinking about events involves mentally simulating different possible scenarios for that event. This means that if an operator has an accurate mental model of a task or system, she or he may be able to solve problems by visualizing a simulation of the task or of system performance. People with low working memory capacity are less likely to construct an accurate mental model from a verbal description, leading to more reasoning errors (Oberauer, Weidenfeld, & Hornig, 2006).
Fiore et al. (2003) emphasize that the connection between the concepts of working memory and mental models “can facilitate a deeper understanding of issues associated with cognitive engineering and decision making research” (p. 508). In this regard, Canas et al. (2003) confirmed that participants’ mental models for a device controlling an electrical circuit relied heavily on visual working memory, by showing that the requirement to maintain a visuospatial memory load (the locations of dots) while making judgments about the circuit was highly interfering, whereas a requirement to maintain a verbal memory load (a set of letters) was not.
Our present understanding of short-term memory is considerably more detailed than it was in the 1960s, when research on short-term memory first began in earnest. Now, we know that short-term memory is not merely a repository for recent events but is closely related to attention and plays a crucial role in many aspects of cognition. Short-term memory is used to temporarily represent and retain information in forms that are useful for problem solving and comprehension. Careful consideration of the short-term and working memory demands imposed by different tasks will ensure better performance in many human factors contexts.
Long-term memories persist from childhood throughout our lives. These memories are qualitatively different from the short-term memories that require continuous rehearsal if they are to persist. Cowan’s (1997) model, which depicts short-term memory as information activated from long-term memory, makes it clear that long-term memory is involved in all aspects of information processing. An operator must often retrieve information from long-term memory to comprehend current system information and to determine what action is appropriate. Failure to remember such things as previous instructions may have catastrophic consequences.
The Japanese air raid at Pearl Harbor on December 7, 1941, one of the worst disasters in U.S. naval history, provides an example. Several factors contributed to the U.S. Navy’s being unprepared for the attack. One important factor was that officers forgot how they were told to interpret events that could signal imminent attack (Janis & Mann, 1977). Five hours before the attack, two U.S. mine sweepers spotted a submarine that they presumed to be Japanese just outside of Pearl Harbor. The sighting was not reported, presumably because the officers had forgotten an explicit warning, given 2 months earlier, that a submarine sighting was extremely dangerous because it could signal the presence of a nearby aircraft carrier. Had the officers remembered the warning, naval forces could have been put on alert and been prepared for the attack that followed. If the information had been presented in a way that enhanced long-term retention of the warning, the ability of the officers to retrieve the information when required could have been improved.
In our discussion of long-term memory, we must distinguish between two tasks that are used to examine its nature: recall and recognition (Danckert & Craik, 2013). In recall tasks, people are presented with information that they have to retrieve later. Many of the experiments we discussed in the context of short-term memory used recall tasks. In recognition tasks, people are presented with a list of items to study. Then they are given a second list and required to indicate for each item whether it was in the original study list. So, whereas recall involves retrieving information from memory, often with no hints, for recognition the studied items are provided.
Until the 1970s, most research on long-term memory was focused on episodic memory, and there is still a lot of interest in this topic. In contrast to the phonological codes of short-term memory, codes in long-term memory were assumed to reflect the meanings of items. For example, in a test of recognition memory, you are given a long list of words to remember; then you are tested a few minutes later with a second list. In that second list, words that were in the first list must be distinguished from those that were not. In such situations, when a word is falsely recognized, that word is often a synonym (that is, a word with a similar meaning) of a word in the original list (Grossman & Eagle, 1970). If the material to be remembered is a passage of text or a specific event, rather than a list of words, only the gist or meaning will be retained rather than any specific wordings used in the text or to describe the event.
Now, we understand that coding in long-term memory is flexible and not restricted to semantic codes. For example, there is some evidence that visual codes exist in long-term memory. In one experiment, people were asked to remember objects (like an umbrella) shown as line drawings. Later, they were shown other drawings of objects that they had seen and some new objects. For objects that had been seen, they responded more quickly when the drawing was identical to the original drawing than when it was not (Frost, 1972). When people are shown and fixate each of a series of objects in a natural scene, their memory is best for the two most recently fixated objects, indicating a role for visual short-term memory. Also, their memory is still well above chance for objects separated by many intervening items, indicating a visual long-term memory component (Hollingworth, 2004). Other evidence shows that concrete and imaginable words (like “umbrella”) are remembered better than abstract words (like “honesty”), apparently because both semantic and visual information can be stored for concrete words but only semantic information for abstract words (Paivio, 1986).
The original modal model claimed that information was transferred from short-term memory into long-term memory through rehearsal (Atkinson & Shiffrin, 1968; Waugh & Norman, 1965). According to the model, the longer information is rehearsed in short-term memory, the greater the probability that it will be transferred into long-term memory. It is a well-established finding that long-term retention of information in fact increases with the number of times that the information is rehearsed (e.g., Hebb, 1961). However, how rehearsal is performed is much more important than how much is done.
We can distinguish between maintenance rehearsal, or rote rehearsal, which involves the covert repetition of material discussed in the previous section, and elaborative rehearsal, which involves relating material together in new ways and integrating the new information with information in long-term memory. Because long-term memory depends on connections between concepts (see below), elaborative rehearsal is much more important for long-term retention than is maintenance rehearsal (Rose, Buchsbaum, & Craik, 2014). Only elaborative rehearsal leads to better performance on recall tests, and although maintenance rehearsal improves performance on recognition tests, it does not do so as much as elaborative rehearsal does (e.g., Woodward, Bjork, & Jongeward, 1973). Even though elaborative rehearsal produces better recognition overall than maintenance rehearsal, it takes time to use the associations established during elaborative rehearsal. Consequently, if a person is pressured to make recognition decisions quickly, much of the benefit of elaborative rehearsal is lost (Benjamin & Bjork, 2000).
Questions about the capacity and duration of long-term memory are difficult to answer. There seem to be no limits on the capacity for acquiring, storing, and retrieving information (Magnussen et al., 2006). Psychologists have debated whether long-term memory is permanent (e.g., Loftus & Loftus, 1980). This is probably an unanswerable question. For many years, forgetting was assumed to reflect the loss of information from memory. Questions focused on whether the loss was due simply to time (decay theory) or to similar events that occurred either before (proactive) or after (retroactive) the events that were to be remembered (interference theory). The results of many experiments were consistent with predictions of interference theory (Postman & Underwood, 1973). However, forgetting in many circumstances is due not to information loss but to a failure to retrieve information that is still available in memory.
Tulving and Pearlstone (1966) had people learn lists of 48 words, 4 from each of 12 categories (e.g., flowers, foods, etc.). During learning, the appropriate category name (flowers) was presented with the words that were to be learned (tulip, daisy, etc.). Subsequently, people were asked to recall the words; half of them did so with the category names provided, and half without. The people provided with the category names recalled more words than those who were not provided with the names. Tulving and Pearlstone concluded that people who were not provided with the category names at recall must have had words available in long-term memory that were not accessible without the category cues.
The point of Tulving and Pearlstone’s (1966) experiment is that effective retrieval cues enhance the accessibility of items in memory. This concept is sometimes called the encoding specificity principle (Tulving & Thomson, 1973): “Specific encoding operations performed on what is perceived determine what is stored, and what is stored determines what retrieval cues are effective in providing access to what is stored” (p. 369). In other words, a cue will be effective to the extent that it matches the encoding performed initially. Appropriate use of retrieval cues to reinstate context is a way to maximize the likelihood that an operator will remember information when it is needed at a later time. Reinstatement of context is particularly important for older adults, who have difficulty retrieving information (Craik & Bialystock, 2006).
The distinction between rote rehearsal and elaborative encoding we mentioned earlier is a difference in processing strategy. Different processing strategies can have drastic effect on long-term retention. Craik and Lockhart (1972) introduced the concept of levels, or depth, of processing. As Craik (2002) notes, “The concept of depth of processing is not hard to grasp—‘deeper’ refers to the analysis of meaning, inference, and implication, in contrast to ‘shallow’ analyses such as surface form, colour, loudness, and brightness” (p. 308). In the levels-of-processing framework, three basic assumptions about memory are required (Zechmeister & Nyberg, 1982). The first is that memories arise from a succession of analyses of increasing depth performed on stimuli. Second, the greater the depth of processing, the stronger the memory, and the better it will be retained. Third, memory improves only by increasing the depth of processing and not by repeating an analysis that has already been performed. The levels-of-processing view leads to the prediction that long-term retention will be a function of the depth of the processing performed during the initial presentation of items.
We examine the influence of levels of processing with “orienting tasks.” An orienting task takes the place of an overt study session and familiarizes people with material that will later be tested. In other words, people perform a particular task on a set of items, unaware that they will receive a subsequent memory test on those items. For example, Hyde and Jenkins (1973) used five types of orienting tasks, two of which apparently required deep-level semantic processing of words’ meanings (rating the pleasantness or unpleasantness of the words, estimating frequency of usage) and three that required shallow processing (checking words for the letters E and G, determining the part of speech for a word, and judging in which of two sentence frames a word fit best). Later recall for the words was better for those people who performed the deep tasks than for those who performed the shallow tasks. Moreover, recall performance on the deep tasks was equivalent to that of a group of people who received standard intentional memory instructions and studied the list without an orienting task. Thus, this study and others indicate that whether or not a person intends to remember the presented information is unimportant. What matters is that the information receives a deep level of processing.
Although the levels-of-processing idea seems to help explain how memory works in some circumstances, it has some limitations. First, the “depth” required for specific orienting tasks cannot be objectively measured. This means that we cannot be certain why recall is better in some conditions than in others. Our explanation can become circular: that is, this task led to better recall than another; therefore it involved a deeper level of processing. Second, another factor, elaboration, also influences retention. Elaboration refers to the number of details provided about material to be remembered. For example, Craik and Tulving (1975) showed that memory for words in complex sentences (e.g., “The great bird swooped down and carried off the struggling – – –”) was better than that for words in simpler sentences (e.g., “He cooked the – – –”), where the word “chicken” could be used to complete either sentence.
Above all, it is the distinctiveness of encoding that is important for memory (Eysenck, 1979; Neath & Brown, 2007). Deeper and more elaborate processing improves retention by producing a representation of an item that is distinct from representations of other to-be-remembered items (Craik, 2002). We can distinguish between information based on the distinctive features of items (item information) and the common information shared by items (relational information; Einstein & Hunt, 1980). Whereas the quality of item information is important for recognition performance, relational information is important for recall. The type of information that is emphasized by particular materials and study strategies will in part determine how well the information is remembered.
Philp, Fields, and Roberts (1989) observed differences in recall and recognition performance for divers who performed memory tests while at surface pressure and during a “dive” in a hyperbaric chamber at a pressure equal to 36 m of seawater. The divers showed a 10% overall decrement in immediate recall of 15-word lists during the dive compared with performance at the surface, and a 50% decrement in a delayed recall test for words from all lists conducted 2 min after the immediate recall test was completed. However, they exhibited no such decrement on a subsequent old-new recognition test of the words. Thus, although the information apparently was intact in the long-term store, the divers had difficulty recalling it. This impairment of free recall suggests that, after people have been in a stressful environment, an accurate assessment of their memories may require that they be provided with cues during debriefing that encourage retrieval of relational information.
The relationship between the type of processing performed during study and the type of memory test is important. Morris, Bransford, and Franks (1977) had people perform either a shallow orienting task (deciding whether a word rhymed with another in a sentence frame) or a deep orienting task (deciding whether a word made sense in a particular sentence frame). The deep orienting task produced better performance on a recognition test for which the unstudied words were semantically similar to the studied words. However, when the new words sounded like the old words, the shallow orienting task led to better recognition. In other words, when the recognition test requires discriminations based on how words sound, it is better to study sound than meaning. This finding illustrates the principle of transfer-appropriate processing: the processing performed during study is effective for memory to the extent that the resulting knowledge is appropriate for the memory test. Lockhart (2002) has suggested that one reason why deep-level processing is usually beneficial to memory is that it increases the likelihood that the processing will turn out to be transfer appropriate to the retrieval contexts encountered later.
Long-term memory can also benefit from the use of strategies that organize the material in meaningful ways. Recall is better for lists of words that come from the same categories than from lists with no obvious category structure, and words from a given category tend to be recalled together even if they were not presented together (Bousfield, 1953). Moreover, studying lists of words grouped in their categories produces considerably better recall than does studying random‑order lists of the same words (Bower, Clark, Lesgold, & Winzenz, 1969). Figure 10.10 shows a conceptual hierarchy for words belonging to the category “minerals.” People studied either a random list of the words in the category or the conceptual hierarchy. More than twice as many words were recalled after studying the conceptual hierarchy. People apparently used the hierarchy to help them retrieve the studied minerals from long-term memory.
FIGURE 10.10Conceptual hierarchy for minerals.
Even for words from different categories, the order in which the words are recalled often shows structure not provided in the study list. In one experiment, younger and older adults (average ages of 20 and 73 years, respectively) were asked to study a list of 20 words and then recall the words in any order (Kahana & Wingfield, 2000). Then they studied the same list of words in a different order, and then made another attempt at recall, until all 20 words were recalled correctly. Recall performance improved on average each time the list was studied, although more slowly for the older adults than for the younger ones. Recalled words tended to be output in organized groups that were relatively constant from episode to episode, even though the order of words in the list was different each time it was studied. Although the older adults took longer to learn the lists, they showed the same level of organization as the younger adults when equated for degree of learning.
The effects of organizational strategies (or the lack of them) on memory performance are apparent not only in the laboratory but also in everyday life. Memory researchers, self-help gurus, and people who have to remember long lists of items have developed many organizational techniques for improving long-term memory. These techniques collectively are mnemonics (Barcroft, 2013; Worthen & Hunt, 2011). A mnemonic is an encoding strategy that organizes material in memorable ways. In addition to providing organization, mnemonic strategies make material more distinctive. This distinctiveness may arise in part from the generation of novel connections between to-be-remembered items induced by the mnemonic technique.
There are two classes of mnemonic techniques: visual and verbal. Visual mnemonics rely on imagery. For this technique, you need to imagine physical relationships between visual images and the items that you want to remember. An example is the method of loci, for which to-be-remembered items are imagined at various locations in a familiar environment, such as your home. When you need to recall the items, you mentally walk through your home to the different locations and “find” the items that you stored there. Visual mnemonics can significantly improve the learning of foreign language vocabulary (Raugh & Atkinson, 1975) and face–name associations (Geiselman, McCloskey, Mossler, & Zielan, 1984).
A verbal mnemonic relates items to be remembered to elements of well-known sentences or stories. Alternatively, the first letters of words to be remembered are combined into new words or phrases (acronyms) or used as first letters of new words in a meaningful sentence (acrostics). So, for example, one well-known verbal mnemonic used to remember taxonomic classification is the sentence “King Philip came over from good Spain,” which represents kingdom, phylum, class, order, family, genus, and species. Both visual and verbal mnemonics can be effective, whether used by themselves or as components in more complex techniques (Cook, 1989).
Mnemonics can be particularly beneficial for elderly people, who may be at higher risk for memory failures (Poon, Walsh-Sweeney, & Fozard, 1980). However, those people who could most benefit from the use of mnemonics, such as the elderly, often forget to use them. Various sentence-based mnemonics have also been proposed to enable users to generate passwords that they will be able to remember but that will be difficult for a hacker to crack (Yang, Li, Chowdhury, Xiong, & Proctor, 2016). For example, think of a sentence that contains at least eight words, and then select a letter, number, or a special character to represent each word. The sentence might be, “I went to London four and a half years ago,” and the resulting password could be iwtl4&ahya. A potential drawback of this approach is that it requires remembering not only the sentence but also the conversions used for each word in the sentence.
So far, we have discussed only internal aids to memory, that is, study and retrieval techniques. Many people also rely on external memory aids, including such things as tying a string around a finger, making notes for reminders, personal digital assistants (PDAs) such as smartphones, and so on. External memory aids are used most often to remember to do things rather than to remember information (Intons-Peterson & Fournier, 1986). One simple strategy explored by Sharps and Price-Sharps (1996) for aiding the memory of older adults was to put a colorful plastic plate in a prominent location in their homes. This plate was used as a place for objects that could be easily lost (keys, glasses, etc.) and for notes to remind the person of future activities. This simple strategy produced a 50% reduction in everyday memory errors for the older adults.
As electronic technology continues to advance, a greater array of commercial memory aids have become available. These range from alarm settings on digital watches to sophisticated smartphone applications. Such memory aids are particularly useful for older adults and patients with memory impairment (Armstrong, McPherson, & Nayar, 2012). For example, Kurlychek (1983) showed that a patient with early Alzheimer’s disease could use an hourly alarm on his watch to consult a written schedule of activities and remind himself what he was supposed to be doing. Because an aid is intended to assist the performance of a particular task performed in a specific context, the effectiveness of an aid depends on the extent to which it corresponds with the situation for which it is being used (Herrmann & Petros, 1990). Inglis et al. (2004) evaluated electronic devices such as PDAs and concluded that the limitations of currently available technology can create difficulty in the use of such devices for memory-impaired and older users. With the widespread use of smartphones, interest has shifted to mobile apps for assisting older adults in remembering to perform tasks such as contacting family members and caregivers. Although several apps are available and progress is being made, “there have been very few successful mobile apps developed for assisting senior adults in their day-to-day activities” (Pang et al., 2015).
COMPREHENDING VERBAL AND NONVERBAL MATERIAL
One important role that long- and short-term memory plays is in understanding, or comprehending, language; that is, in reading and listening. Reading printed text can involve something as simple as reading the word “stop” on a sign or as complex as reading a technical manual providing instructions for operating a system. Similarly, spoken language can be a single utterance (the word “go”) or a complex narrative. The comprehension and retention of a message are important for any situation involving verbal material. In this section, we examine the interplay between semantic memory (our storehouse of general knowledge) and the processes involved in the comprehension and retention of sentences, text, and structured discourse.
To understand comprehension, we need to understand the processes involved in semantic memory (Tulving & Donaldson, 1972). Such processes include how knowledge is represented and how this knowledge is accessed (see McNamara, 2013, for a review). We know that people can judge faster whether an object is a member of a category when the category size is small rather than large. For example, the decision that a beagle is a dog can be made faster than the decision that a beagle is an animal. We call this the category size effect. Also, decisions about category membership can be made faster when the object is a typical member of the category than when it is less typical. For example, the decision that a canary is a bird can be made faster than the decision that a buzzard is a bird. We call this the typicality effect.
Two types of semantic memory models have been proposed to explain the category size and typicality effects. Network models assume that concepts are represented by distinct nodes that are interconnected within an organized network (see Figure 10.11). In a network model, the time to verify a sentence is a function of the distance between the concept nodes and the strengths of the connecting links between them. For example, Collins and Loftus (1975) proposed a spreading-activation model using two separate networks. In the conceptual network, a fragment of which is shown in Figure 10.11, concepts are organized according to semantic similarity. In the lexical network, the labels for each concept are organized according to auditory similarity. Retrieval in both networks occurs as the activation of one concept or name spreads through the network along connecting links, with nodes farther from the concept receiving weaker activation at longer delays than nodes that are close to the concept.
FIGURE 10.11A representation of concept relatedness in a connected network of concepts.
The second kind of model relies on feature comparisons. These models propose that concepts are stored as lists of semantic features, and verification occurs when the object and category in a statement share matching features. For example, in Smith, Shoben, and Rips’s (1974) feature-comparison model, all features of the object and category are compared to yield an overall similarity value. When the similarity is very high or very low, a fast true or false response can be made. When the similarity is ambiguous, only the most important, defining features of the object and category are closely examined. The true or false response is then based on whether this subset of features matches or not.
Both network and feature models have provided good explanations of the category size and typicality effects. They have also been very useful in applied settings. Models like these have been used to structure knowledge data bases, such as those used for expert systems and decision-support systems (see Chapters 11 and 12).
Although network and feature models explain the category size and typicality effects in category verification, other important models can explain a wider range of data. Two such models include distributed network models and high-dimensional spatial models. Distributed network models are distinguished from the network models presented above in that concepts are not represented by distinct nodes but by patterns of activation across many nodes in the network. In distributed networks, knowledge is represented in the weights of the connections between nodes, which determine the patterns of activation induced by different stimuli. Distributed networks learn by applying rules to adjust the connection weights between nodes: The output of the network for a particular stimulus is compared with the desired output for that stimulus, and the weights are adjusted to reduce the mismatch between the actual and the desired output. Models of this type can explain many of the findings obtained from tasks relying on semantic memory (e.g., Kawamoto, Farrar, & Kello, 1994).
High-dimensional spatial models are similar to distributed network models in that concepts are represented as points in a multidimensional space. The most well-known model of this type uses Latent Semantic Analysis (LSA; Günther, Dudschig, & Kaup, 2015; Landauer, 1998; Landauer, McNamara, Dennis, & Kintsch, 2007). The basic idea behind LSA is that similarities between words (concepts) can be inferred from their co-occurrences in written text. When LSA is applied to a sample of text, it produces a semantic space in which different concepts cluster together. One analysis examined more than 90,000 words in over 37,000 different contexts representing material that an English reader might read from an early grade through the first year in college. The resulting LSA representation is a semantic network of a first-year college student.
It is surprising that a representation constructed solely on how words co-occur can be at all useful. However, LSA has yielded accurate simulations of many language phenomena and allowed automation of many tasks involving language. One of the more impressive applications of LSA has been the automatic grading of essays (Landauer, Laham, & Foltz, 2003). For this application, LSA is applied to a large sample of texts dealing with the topic of the essays. A representative sample of essays is then scored by human graders. Those graded essays and the essays to be graded automatically are represented as LSA vectors. The similarities between each ungraded essay and each previously graded essay are computed, and these similarities are used to assign grades to each essay. It has been demonstrated that the grades assigned automatically to the essays are as accurate as those assigned by a human. Landauer et al. suggest that one immediate application of this technology is in writing tutorial systems. Also, many standardized admissions tests, such as the Scholastic Aptitude Test (SAT) and Graduate Record Examination (GRE), now require an essay, and an automated system for grading those essays could greatly reduce the time necessary for grading and standardizing the results.
Reading is a complex process that requires people to retrieve information from semantic memory and integrate information across sentences and passages in a text. Reading efficiency is affected by many factors. These include the purpose for reading, the nature of the materials, the educational background of the reader, and characteristics of the environment. In situations where the successful operation of a system depends on the operator’s ability to read effectively, the human factors professional must consider these factors and how best to present written material. According to Wright (1988, p. 265), “As working life becomes more information intensive, so a better understanding of how to design and manage written information becomes more urgent.” One study, looking at information in instruction booklets, recommended that written information be evaluated by way of usability testing prior to implementation (Brooke, Isherwood, Herbert, Raynor, & Knapp, 2012).
Many situations in which an operator must read information also require that she read it quickly. Reading speed is influenced by several factors, including the complexity of sentences and the goals of the reader. A sentence can be decomposed into some number of basic ideas, or propositions. The more propositions there are, the longer it takes to read the sentence, even if the number of words is held constant (Kintsch & Keenan, 1973). If the reader’s goal is to remember a sentence word for word, it will take longer to read it than if the reader’s goal is comprehension (Aaronson & Scarborough, 1976).
Reading speed is irrelevant if the information is comprehended poorly. Comprehension can be improved by making changes to the syntactic structure of the material. For example, comprehension is better when relative pronouns (e.g., that, which, whom) are used to signal the start of a phrase than when they are omitted (Fodor & Garrett, 1967). Consider the sentence, “The barge floated down the river sank.” Such a sentence is often called a garden path sentence, because its construction leads the reader to interpret the word floated as the verb for the sentence, but this interpretation must be revised when the word sank is encountered. Changing the sentence to read “The barge that floated down the river sank” eliminates this ambiguity. However, not all garden path sentences can be remedied in this way. For instance, consider the sentence “The man who whistled tunes pianos.” This sentence will be read more slowly and comprehended less well than the equivalent sentence “The whistling man tunes pianos,” in which the word tunes is unambiguously a verb.
The effects that different syntactic structures have on reading comprehension have been used to construct theories of how people read and represent material (Frazier & Clifton, 1996). These theories focus on a variety of syntactic features, including word order and phrase structure rules (templates for possible configurations of words in phrases) and the case of a noun (which is related to the role it plays in the sentence structure; Clifton & Duffy, 2001). Sentences that contain nested clauses, such as “The man from whom the thief stole a watch called the police,” are more difficult to comprehend than sentences that do not contain nested clauses (Schwartz, Sparkman, & Deese, 1970). This finding has been used to study the contribution of working memory limitations in reading comprehension.
Semantic structure is as important as syntactic structure. Some words seem to require more effort to understand than other words in some contexts. For example, the word kicked in the sentence “The man kicked the little bundle of fur for a long time to see if it was alive” takes longer to process than the word examined in the sentence “The man examined the little bundle of fur for a long time to see if it was alive” (Piñango, Zuriff, & Jackendoff, 1999). One explanation for this finding is that the verb examine implies an act that takes an extended period of time, consistently with the phrase for a long time. The verb kick must be changed from an act that is usually brief to an act that is repeated over an extended period of time to be consistent with the phrase for a long time. Changing the representation of the verb kick requires time and effort.
Another example demonstrating the importance of semantic structure can be seen in how events are ordered in a sentence. A sentence will be comprehended best when the order of events in the sentence follows the actual order of events (Clark & Clark, 1968). This has implications for the way that written instructions should be presented. Dixon (1982) presented people with multistep directions for operating an electronic device and measured the time it took for them to read each sentence. Reading times were shortest when the action to be performed (e.g., “turn the left knob”) came before the specific details describing the desired outcome of the action (e.g., “the alpha meter should read 20”). This finding suggests that comprehension will be easiest when instructions are organized around the actions to be performed.
Complex communication involves integrating information across many sentences. Successful readers construct an abstract, rather than literal, representation of what is read. These readers make inferences about relations and events that are implied by the text but not directly stated. These inferences will later be remembered as part of the material that was read (Johnson, Bransford, & Solomon, 1973). Working memory plays an important role in comprehension because working memory allows new information to be interpreted in terms of and integrated with information and inferences already in memory. Poor readers differ from good readers primarily in the efficiency with which the integration of new propositions into the working representation can be performed (Petros, Bentz, Hammes, & Zehr, 1990).
Readers seem to form representations of material in the form of organized structures called schemas (Rumelhart & Norman, 1988; Thorndyke, 1984). Schemas are frameworks that organize our general knowledge about familiar objects, situations, events, actions, and sequences of events and actions. Most importantly, schemas cause a person to expect certain events to occur in certain contexts. These expectancies make interpretation of information easier and help the reader determine its relative importance (Brewer & Lichtenstein, 1981).
A famous experiment performed by Bransford and Johnson (1972) demonstrated the importance of an appropriate schema in text comprehension. Students read the passage shown in Table 10.1, then rated its comprehensibility and tried to recall the ideas contained in it. Students who were told prior to reading the passage that the topic was “washing clothes” rated comprehensibility higher and remembered many more of the details than did students who were not told the topic in advance.
TABLE 10.1
Passage about the Topic of Washing Clothes
The procedure is actually quite simple. First you arrange things into different groups. Of course, one pile may be sufficient depending on how much there is to do. If you have to go somewhere else due to lack of facilities, this is the next step; otherwise you are pretty well set. It is important not to overdo things. That is, it is better to do too few things at once than too many. In the short run this may not seem important, but complications can easily arise. A mistake can be expensive as well. At first the whole procedure will seem complicated. Soon, however, it will become just another facet of life. It is difficult to foresee any end to the necessity for this task in the immediate future, but then one can never tell. After this procedure is completed, one arranges the material into different groups again. Then they can be put into their more appropriate places. Eventually they will be used once more and the whole cycle will then have to be repeated. However, that is part of life. |
Similarly, schemas for technical documents can aid in the comprehension and retention of technical material. Because failure to heed warnings may lead to injury, we should design instruction manual warnings around familiar schemas to enhance comprehensibility and memorability.
Young and Wogalter (1990) noted that comprehension and memory could be improved by increasing the likelihood that the warning is noticed in the first place, and then by providing visual information to accompany the verbal information. They conducted two experiments, one using instruction manuals for a gas-powered generator and the other instruction manuals for a natural‑gas oven/range, in which the typeface of the warning messages was either plain or salient (larger type, orange shading) and the messages were either accompanied by pictorial icons or not (see Figure 10.12). Comprehension and memory of the warnings were significantly better when they were presented in salient type and included a pictorial icon. The authors suggest that this effectiveness of the salient print and icon combination may be due to better integration of the verbal and visual codes.
FIGURE 10.12Warnings differing in conspicuousness of print and the presence of an icon.
Teams of operators communicate with each other using speech to coordinate their performance. In many organizations, team leaders provide briefings to their team members at the beginning of the work shift. The points we just made about syntax, semantics, and schemas for reading comprehension also apply to the comprehension of spoken language (Jones, Morris, & Quayle, 1987). Also important for comprehension of spoken language is “prosodic phrasing,” or the grouping together of words by their tonal pitch, duration, and rhythm (Frazier, Carlson, & Clifton, 2006). Box 10.1 provides a vivid example of the importance of spoken and written communication in a real-life example.
Research on speech comprehension has been conducted by studying conversation, in which two or more people take turns conveying information to each other. When a person assumes the role of a listener, we assume that he or she tries to understand what the speaker wants to convey. Consequently, the listener assumes that what the speaker says is sensible and constructs an interpretation that hopefully is the one intended by the speaker.
Several components of a spoken utterance can be distinguished (Miller & Glucksberg, 1988). These include the utterance itself, the literal meaning of the utterance, and the meaning intended by the speaker. Comprehension requires more than just establishing the literal meaning; the speaker’s intention also must be inferred. Many communication errors occur because a listener misinterprets a speaker’s intentions.
The listener uses a number of rules to establish both the literal and the intended meaning of an utterance. According to Grice (1975), the most important rule is the cooperative principle: the assumption that the speaker is being cooperative and sincere, and is trying to further the purpose of the conversation. The cooperative principle specifies several conversational rules, or maxims. These maxims can be classified by conversational quantity, quality, relation, and manner (see Table 10.2). Overall, these maxims allow the listener to assume that the speaker is making relevant and unambiguous statements that are informative and truthful. If these maxims hold, then the listener’s task of constructing an intended meaning from the speaker’s literal meaning is made as easy as possible.
TABLE 10.2
Grice’s Conversational Maxims
Maxims of Quantity |
• Make your contribution as informative as is required. |
• Do not make your contribution more informative than is required. |
Maxims of Quality |
• Try to make your contribution one that is true. |
• Do not say what you believe to be false. |
• Do not say that for which you lack adequate evidence. |
Maxim of Relation |
• Be relevant. |
Maxims of Manner |
• Be perspicuous. |
• Avoid obscurity of expression. |
• Avoid ambiguity. |
• Be brief. |
• Be orderly. |
A speaker can improve a listener’s comprehension by making direct references to information that was provided earlier in the conversation. Listeners seem to make use of a given-new strategy (Haviland & Clark, 1974). This strategy identifies two types of information in any utterance: given and new. The given information is assumed by the speaker to be already known by the listener, whereas the new information is to be added by the listener to the old. If the speaker can arrange sentences to distinguish between given and new information, the listener’s task can be made easier.
The literal meaning of some utterances is not the same as the meaning intended by the speaker. This occurs for indirect requests, in which a request for some action on the listener’s part is not stated directly. For example, the question “Do you know what time it is?” contains an implicit request for the listener to provide the speaker with the time of day. Much of spoken language is figurative and includes expressions that use irony, metaphor, and idiomatic phrases. An example of an idiomatic phrase that also uses metaphor is “making a mountain out of a molehill.” In a case like this, the listener appears to construct both the literal and the nonliteral meaning of the utterance. The speed with which the listener comprehends figurative meanings of utterances is usually slower than the speed with which the listener comprehends their literal meaning (Miller & Glucksberg, 1988). One implication of this research is that figurative speech should be avoided in communication with operators unless the meaning is very clearly dictated by the context.
It will probably come as no surprise to learn that in group problem-solving situations, speech is the best way for group members to communicate. Chapanis, Parrish, Ochsman, and Weeks (1977) had two-member teams solve an equipment assembly problem using one of four modes of communication: typewriting, handwriting, speech alone, and a condition where communication was not restricted to any particular mode. The problem was solved approximately twice as fast in the speech alone and unrestricted conditions than in the typewriting and handwriting conditions. This finding held even though the communications in the speech and unrestricted conditions were much lengthier. One of the reasons for the better performance in these conditions was that the teams could engage in problem solving and communication activities simultaneously, whereas they could not do so in the typewriting and handwriting conditions.
BOX 10.1TEXT COMPREHENSION AND COMMUNICATION DURING AN AIR DISASTER
On the afternoon of July 19, 1989, United Airlines flight 232 took off from Denver. While en route to Chicago, the plane, a DC-10, experienced a catastrophic failure of its center engine. Debris from the engine collided with the tail of the aircraft, disabling not only the primary hydraulics system but the second and third backup systems as well. The flight control system was rendered inoperable, and the crew could only make right turns (and these only with great difficulty). The plane crash-landed in Sioux City, Iowa, using only the throttles, which operate much like the accelerator pedal in a car. The plane was torn apart in the landing, but 184 of the 292 passengers and crew survived.
There are two aspects of the crash of flight UA-232 that deserve consideration in our discussion of verbal and written communication. The first is the communications that the flight engineer had with the ground maintenance crew, and the second is the role of the flight manual during the emergency.
In the early stages of the disaster, the flight engineer, Dudley Dvorak, attempted to convey to the ground maintenance crew that all hydraulic systems were lost. We have to understand that although it was obvious to the crew on the flight deck that all three systems had failed, the ground crew persisted in interpreting Dvorak’s messages from the perspective that a failure of three completely independent systems was impossible. In other words, the ground crew’s schema did not allow for the possibility that all three systems could fail, which prevented them from understanding what had happened.
Dvorak’s first contact with maintenance was to report that the number two engine was gone and that they had “lost all hydraulics.” Maintenance responded to Dvorak by asking about the number two engine. Then maintenance attempted to confirm that (hydraulic) systems one and three were operating normally; Dvorak replied, “Negative. All hydraulics are lost. All hydraulic systems are lost.” Even after this exchange, maintenance asked about hydraulic fluid levels. Dvorak reported back that there was no hydraulic fluid left. The maintenance crew then consulted the flight manual, and several more minutes of confused discussion took place between Dvorak and the maintenance crew before it was established that not a single one of the hydraulic systems was operational.
In Captain Haynes’s words, “The hardest problem that Dudley had was convincing them that we didn’t have any hydraulics. ‘Oh, you lost number two,’ ‘No, we lost all three,’ ‘Oh you lost number three,’ ‘No, we’ve lost all of them,’ ‘Well, number one and two work,’ ‘No,’ well we went on like this for quite a while” (http://yarchive.net/air/airliners/dc10_sioux_city.html).
The flight manual, to which the ground maintenance crew referred in an attempt to help recover the plane, is a book provided by the manufacturer of the aircraft. It contains descriptions of possible scenarios and the procedures appropriate for each. On most aircraft the flight engineer also has an abridged version of the flight manual, which can be consulted in the event of an unusual situation or an emergency. Before the ground maintenance crew began searching the manual (and logbooks and computer databases) for a solution to the hydraulics failure, Dvorak had consulted the manual for a solution to the engine failure. Emergency procedures in the manual are written in checklist form, without explanation. This reduces the demands on memory and problem-solving resources during an emergency. For example, the manual said that the first thing the flight crew should do is close the throttle to the failed engine. After that, the fuel is to be shut off. However, because of the hydraulics failure, these controls did not respond appropriately, signaling to the crew that something else had gone very badly wrong.
There were no procedures provided for total hydraulic failure in the flight manual, because the manufacturer believed that by engineering three such systems, each independent of the others, such a failure was impossible. Therefore, there was nothing for the ground maintenance crew to find in the manual. Because the maintenance crew kept making suggestions that the crew had already tried, as well as suggestions that required an operational hydraulics system, in frustration Captain Haynes instructed Dvorak to cease communications with maintenance. In the final analysis, there really was nothing that maintenance could have done for the crew.
When Captain Haynes describes his experiences on flight UA-232, he is careful to emphasize that one of the factors that helped him and his crew to land the plane and save so many of the passengers was communication: the flow of information between the members of the crew, the communications between the Captain and the air traffic controllers at Sioux City, and the communications between the emergency teams on the ground in Sioux City. Verbal and written forms of communication were an integral part of the “success” of flight UA-232.
We should note that experts who investigated the crash of flight UA-232 deemed it impossible to successfully land a plane in a simulator under the conditions experienced by the crew. For this reason, Captain Haynes and his crew were commended for their extraordinary performance and credited with saving the lives of the 184 survivors. Captain Haynes attributes the crew's outstanding performance to a management technique called crew resource management, which encourages all members of a team to communicate with the team leader in solving problems.
An important concept within the field of human factors that extends across issues of memory and comprehension is that of situational awareness (Endsley & Jones, 2012). Situation awareness is defined as “the perception of the elements in the environment …, the comprehension of their meaning, and the projection of their status in the near future” (Endsley, 1988, p. 97). The term awareness emphasizes the importance of working memory, particularly the part that Baddeley (2000) calls the episodic buffer, which in part determines consciousness and awareness. The things of which we are aware are affected by attentional factors (i.e., the central executive), which suggests that our situational awareness will be limited if our attention is directed to inappropriate elements of the environment. Endsley and Jones indicate that a person’s failures of selective attention (such as those that might occur when talking on a mobile phone and driving) may restrict his or her situational awareness.
Loss of situational awareness is of particular concern when tasks are automated. Consider, for example, partially or fully autonomous cars in which the driver's responsibilities are reduced. Because automation of driving will only work during very predictable situations, such as highway driving, the driver must be prepared to take over when the situation changes unexpectedly and automation fails. Maintaining a level of awareness that permits the driver to re-engage attention when immediate action is needed is a topic of ongoing investigation in human factors (Sirkin, Martelaro, Johns, & Ju, 2017).
Because situational awareness depends on working memory, it will be limited by the same factors that limit working memory capacity and accuracy. Furthermore, a person’s level of mental workload may also influence his or her level of situation awareness. If a person’s mental workload is very high, his or her situation awareness may be poor. However, even when a person’s mental workload is within acceptable limits, his or her situation awareness may still be poor. That is, comprehension of events in the environment may be limited even if the person’s attentional and memory resources are not being overloaded.
Situation awareness is often measured by asking an operator to perform a primary task such as driving or flying in a simulated environment. We evaluate situation awareness by measuring the operator’s immediate memory and understanding of the objects and events in the task, and his or her ability to predict the future behavior of the system. As with mental workload, there are both subjective and objective measures of situation awareness. Subjective measures require that the operator or an expert observer rate the operator’s awareness for a specified scenario or period. Objective measures probe the operator for information about some aspect of the task.
Two accepted methods for obtaining objective measures (Vu & Chiappe, 2015) include Situation Awareness Global Assessment Technique (SAGAT; Endsley & Jones, 2012) and Situation Present Assessment Method (SPAM; Durso & Dattel, 2004). For SAGAT, the simulation is frozen and the operator is asked several questions; for example, the experimenter may stop a simulated driving task and ask the driver to recall where the other vehicles on the roadway were located, to predict what a vehicle at the side of the road will do, and so on. The accuracy of the responses to the questions is taken as an indicator of the extent of the operator’s conscious awareness of the situation. For SPAM, on the other hand, questions are presented one at a time during the task, more like a dual-task procedure for measuring mental workload. To preclude interference from high momentary workload, the question is presented only after the operator has responded affirmatively in response to a prompt that she is ready to receive the question. SPAM allows use of response time as well as accuracy in the assessment of situational awareness within the ongoing dynamic task. In addition, the processes that the operator is using to maintain awareness, for example, off-loading subtasks to aids such as electronic navigation devices. The issues involved in measuring situation awareness are similar to those discussed for mental workload in Chapter 9—intrusiveness, ease of implementation, operator acceptance, and so forth—and consideration should be given to these issues when selecting a particular method for use.
Successful performance of virtually any task depends on memory. If accurate information is not retrieved at the appropriate time during the performance of a task, errors may occur. Three general categories of memories can be distinguished on the basis of their durability. Sensory memories retain information in a modality-specific format for very brief periods of time. Short-term memories are retained in an active state and are used for reasoning and comprehension. Long-term memories are outside of awareness and not in a highly activated state but may be retained for extended periods of time. As our knowledge about memory has progressed, our view of memory has evolved into one of flexible, dynamic systems with multiple coding formats. The characteristics of different kinds of memory and the processes in which they are involved have predictable effects on human performance, which we need to remember while designing human–machine systems.
Memory is intimately involved in the comprehension and communication of information. Verbal materials and environmental events are identified by accessing knowledge in semantic memory. Comprehension of both written and spoken language is based on mental representations of the information being conveyed. These representations are constructed from the individual’s perception of the material and the context in which it is perceived. To the extent that information is consistent with the observer’s mental representation, language comprehension will be facilitated. Nonverbal events must also be comprehended, and the concept of situation awareness emphasizes the importance of accurate comprehension for the operation of complex systems. Memory and comprehension, along with perception and attention, play roles in situation awareness, which is assessed nowadays by human factors specialists in a variety of settings. They also play pivotal roles in thought and decision making, the topics of our next chapter.
Baddeley, A. (1999). Essentials of Human Memory. Hove, UK: Psychology Press.
Baddeley, A., Eysenck, M. W., & Anderson, M. C. (2009). Memory. Hove, UK: Psychology Press.
Kahana, M. J. (2012). Foundations of Human Memory. New York: Oxford University Press.
Marsh, E., McDermott, K. B., & Roediger, H. L., III (Eds.) (2006). Human Memory: Key Readings. New York: Psychology Press.
Neath, I., and Surprenant, A. (2003). Human Memory: An Introduction to Research, Data, and Theory (2nd ed.). Belmont, CA: Wadsworth.
Radvansky, G. A. (2011). Human Memory (2nd ed.). New York: Routledge.
Rayner, K., Pollatsek, A., Ashby, J., & Clifton, C., Jr. (2012). The Psychology of Reading (2nd ed.). New York: Psychology Press.
Tulving, E., & Craik, F. I. M. (Eds.) (2000). The Oxford Handbook of Memory. Oxford, UK: Oxford University Press.
Notes