17Multimodal resources and the organization of social interaction
Abstract: Human communication involves not only language but, crucially, the body – being fundamentally multimodal (that is: involving language, gesture, gaze, body postures, movements, objects manipulations and arrangements of bodies in space). This chapter explores some contemporary issues in this field, elaborating on various challenges this perspectives opens both methodologically and theoretically. The study of langauge and the body in social interactions is grounded on video recordings of naturally occurring interactions and on transcripts and other forms of annotation. Analytically, this rich documentation makes possible to study the complexity of the moment-by-moment sequential and temporal organization of the emerging interaction. Building on conversation analysis, the chapter shows how multimodality contributes to the study of turn taking, sequence and action.
Keywords: social interaction, multimodality, conversation analysis, language, body, video, sequentiality
1Introduction
Social interaction, and human communication more generally, mobilizes a range of linguistic and embodied resources through which participants achieve the intelligibility of their actions and build their intersubjective interpretations. Although language is a key means of communication, it is reductive to limit the study of communication to linguistic aspects – given the fundamental contribution of the body to face-to-face interaction. A contemporary challenge consists in developing an integrated conception of language and the body in communicative action – without favouring a priori one over the other, but by looking at the way they are specifically and situatedly mobilized together by participants in social interaction. Thus, this chapter explores some ways in which multimodal resources comprising different linguistic manifestations (from grammar to prosody) and embodied ones (gesture, gaze, head movements, facial expressions, body postures, body movements, etc.) organize social interaction. The chapter refers to contemporary research on gesture and embodiment and focuses more particularly on the contribution of conversation analysis to this literature and specifically on studies of social interaction based on video recordings of naturally occurring social activities.
In face-to-face interaction, participants not only speak together but also gesticulate and move their bodies in meaningful and coordinated ways. Gesture studies have shown that gestures in conversation are generated by the same processes that produce talk (Kendon 1980; McNeill 1985). Made predominantly by speakers but strongly oriented to their partners (Schegloff 1984), gestures are finely synchronized with the structure of discourse (Müller 1989) and of talk in interaction (Bohle 2001; Kendon 2004); moreover, they are finely tuned with the conduct of the co-participants to whom they are addressed. This has prompted gesture studies to investigate ‘interactive gestures’ (Bavelas et al. 1992) in dialogue – that is, gestures that do not refer to the topics at hand but instead refer to the interlocutor, monitoring shared understanding and establishing common ground (Clark 1996), seeking agreement and also maintaining conversation and regulating turns at talk. These gestures, which typically take the form of either pointing towards the interlocutor or more complex hand shapes such as an exposed palm, an offering open hand, etc., belong to the range of visibly-embodied resources that participants mobilize in order to build the systematic order of social interaction.
Interest in how human interaction works in its ordinary social as well as in its professional and institutional settings has prompted the study of video recordings of naturally occurring activities aimed at understanding how participants smoothly achieve the finely-tuned complex coordination of their actions. On the basis of naturalistic data, conversation analysis, inspired by Garfinkel’s ethnomethodology (1967), has focused on human interaction as endogenously and methodically organized. Interaction is here considered not as being governed by external norms and rules but as being locally achieved by participants, based on micro-practices that are both context-free and context-shaped (Heritage 1984) – such as practices for self-selecting, for beginning a new turn, for recognizing transition-relevance points (Lerner 2003; Sacks, Schegloff & Jefferson 1974), for repairing troubles (Schegloff, Jefferson & Sacks 1977), etc. These practices are organized by participants in a publicly accountable way, that is, in a way, which is intelligibly produced for and interpreted by them as action unfolds in real time. This public accountability is built through the mobilization of a range of resources. These include language and gesture and also integrate other aspects of bodily conduct, such as body postures and movements. The booming literature on multimodal resources (multimodality being conceived in this perspective as comprising language, gesture, gaze, head movements, facial expressions, body postures and – increasingly – object manipulation and technology and body movements within space) shows that participants exploit both conventional forms and improvised and occasioned means to produce the intelligibility of their actions and coordinate with the actions of others.
This review begins with a methodological note on the use of video technologies for the study of social interaction (2.). It then develops the notion of multimodal resources (3.) and shows how they contribute to the organization of social interaction (4.). The chapter also shows how it is possible to enlarge the classical view of gesture and the body by taking into consideration the material (5.), spatial (6.) and mobile (7.) dimensions of embodiment. Finally, the chapter offers an exemplary analysis of a piece of data (8.), focusing on analytical and methodological issues raised by practices of pointing and referring in interaction.
2The naturalistic use of audio and video recording technologies
The detailed consideration of multimodal resources (gestures, gazes, head movements, facial expressions, body postures, manipulations of objects, etc.) depends in a crucial way on the technologies for documenting social action. Even if film potentialities were exploited by social and anthropological research as soon as the first technological devices for creating moving images were available (see Mondada 2012), their systematic exploitation for the study of human communication properly began in the 1950s when technological possibilities began to be used within new paradigms studying social interaction and human communication in naturalistic settings.
In this context, an important role in the use of film for social research was played by an interdisciplinary team at the Veterans Administration Hospital of Palo Alto, lead by Bateson – who had been using films in Bali in collaboration with Mead in the 1940s – using film recordings for the study of communication in psychotherapy and in families with a member affected by schizophrenia. In 1955, this group worked together on the video recording of a psychiatric interview between Bateson and one of his patients, Doris, then transcribed by Hockett, Birdwhistell and McQuown (see McQuown 1971). The title of the project, Natural History of an Interview, significantly refers to an analysis of human behaviour that recognizes the importance of ‘spontaneous conversational materials’ in ‘a variety of contexts’ (McQuown 1971, ch. 10: 9, 11). On the basis of this film, Birdwhistell eventually developed his famous analysis of the cigarette scene and the discipline of kinesics (1970). The Natural History of an Interview can be seen as inaugurating the contemporary use of video in the social sciences. A few years later, influenced by Scheflen (1972), the founder of context analysis and a member of the Palo Alto group, Kendon (1967, 1970, 1990) gave new vigour to the study of gesture. Some years afterwards, various praxeologically and pragmatically oriented approaches developed more and more sophisticated ways of documenting social practices, and the use of video began to spread in all the social and human sciences, not only in anthropology, where it had been used since its invention, but also in linguistics, sociology, studies of work, technology studies and education.
Conversation analysis developed from the 1960s on, on the basis of audio-video recordings documenting naturally occurring practices. Although a majority of the first studies in conversation analysis used audio recordings and focused on telephone conversations – a form of interaction in which participants were not relying on their mutual visual access –, the use of video began very early on. As early as 1970, in Philadelphia, Charles and Marjorie Harness Goodwin carried out film recordings of everyday dinner conversations and other social encounters. After 1973, these recordings were used by Jefferson, Sacks and Schegloff in research seminars as well as in published papers. In 1975, at the annual meeting of the American Anthropological Association, Schegloff presented a paper co-authored with Sacks, who had been killed a few weeks earlier in a car accident, on ‘home position’ (Sacks & Schegloff 2002). This was an early attempt to describe bodily action systematically (see also Schegloff 1984). In 1977, Charles Goodwin presented his dissertation at the Annenberg School of Communications of Philadelphia (later published as Goodwin 1981). The dissertation was based on approximately 50 hours of videotaped conversations in various settings (Goodwin 1981: 33).
Early work by Charles Goodwin (1981) in the USA, as well as by Christian Heath (1986) in the UK, extensively used film materials in order to analyse and understand how, in co-present interaction, humans orderly and situatedly mobilize a large range of verbal, sound and visual resources in order to produce intelligible – accountable – actions, as well as to interpret publicly displayed and mutually available actions (Streeck 1993). In an important way, this early work was convergent with some of the assumptions made by pioneers in gesture studies, such as Kendon (1990) and McNeill (1985) who had argued that gesture and talk are not separated ‘modules’ for communication but originate from the very same linguistic, cognitive and social mechanisms as speech.
Further developments in conversation analysis have been characterized by an increasing interest in data gathered in institutional and professional settings, opening up a program of comparative studies of speech exchange systems differing from conversation and characterized by distinctive and restrictive speaker’s rights and obligations (cf. Drew & Heritage 1992: 19; Sacks, Schegloff & Jefferson 1974: 729) as well as by complex activities, workspaces and bodily arrangements of participants (Heath & Luff 2000).
The documentation of such complex situations has implemented new ways of recording data, prompting a reflection on the way in which data are to be collected (Heath, Hindmarsh & Luff 2010; Mondada 2006, 2012) in order to document the complexity of embodied action, including not only gestures but also object manipulations, uses of documents, uses of technologies, as well as complex forms of multi-party interaction, beyond mutually focused encounters and within peculiar ecologies and environments – all demanding peculiar techniques of recording and often more than one camera.
3Multimodal resources
The use of video technologies allows the researcher to conduct a detailed examination of the way in which action gets situatedly organized thanks to the mobilization of a multiplicity of multimodal resources.
The notion of ‘resource’ covers both conventional forms – such as grammar – and less standardized and more opportunistic means that are used by participants to build the intersubjective accountability of their actions. Thus, the notion of ‘resource’ invites the researcher to take into consideration the indexicality of linguistic resources, as well as the systematic and methodic use of embodied resources. This avoids reifying certain well-studied (mostly linguistic) resources and ignoring other less-studied ones or extracting resources from the context in which they are situated. The value and meaning of a resource is context-dependent, being related both to the sequential organization of social interaction and to the situated occasion of its use. In return, a resource also reflexively shapes the particular interpretation of the context that is made relevant at that particular moment. Moreover, the contextually specific use of a resource might also shape its form and intelligibility as a form that will be available in the future – thus prompting semiotic change (Mondada 2014).
Very early on, the very precise temporality of linguistic and gestural resources was described by gesture studies (Seyfeddinipur & Gullberg 2014). For example, speaker’s gestures generally slightly precede their lexical affiliates (Kendon 1980; McNeill 1992; Schegloff 1984). Their peculiar timing is the result of interactive work by which talk and gesture and, more globally, talk and posture are organized in a way that aligns them temporally, for example either by delaying talk to adjust to gesture or the reverse (Condon 1971; Kendon 2004: 135). Co-occurrence of gesture and speech has been widely documented in gesture studies (McNeill 1985), treating talk and gesture as ‘composite signals’ (Clark 1996: 156), as belonging to an ‘integrated message model’ and by showing, for example, that gesture and facial displays are used simultaneously with words, being mobilized together to produce ‘visible acts of meaning’ (Bavelas & Chovil 2000) or ‘visible action as utterance’ (Kendon 2004). However, despite these studies, the way in which multimodal resources as a whole (meaning a wider range of embodied resources than gesture only) are mobilized within multiple temporal and sequential relationships in situated interactions remains to be systematically investigated by focusing not only on the speaker but also on the actions of the addressees and on the entire participation framework (Goodwin 1981, 2000, 2007a).
This enlarged approach to multimodality – considered as the integrated study of all the relevant linguistic, embodied and material resources participants exploit for organizing social interaction in an audible-visible intelligible way – has been developed more particularly by work in conversation analysis (Mondada 2014).
On the one hand, this has prompted systematic investigations focused on a particular multimodal detail; for example, Schegloff (1984) studies gestures produced by speakers, Goodwin (1981) divergent vs. mutual gaze prompting re-starts at the beginning of the turn, Mondada (2007b) pointing as displaying an imminent speaker’s self-selection; Stivers (2008) nods as expressing affiliation in storytelling and Peräkyla and Ruusuvuori (2006) facial expressions as manifesting alignment and affiliation in assessment sequences.
On the other hand, a complementary approach considers the coherent and coordinated complexity of various embodied conducts together; for example, Heath (1989) studies together gaze, body posture and body manipulations; Streeck (1993) gesture and gaze and Mondada and Schmitt (2010) and Hausendorf, Mondada and Schmitt (2012) the coordination of a range of multimodal resources, going from gesture to body position and considering the distribution of bodies in space. This emphasis on complex multimodal gestalts also invites researchers to investigate the entire body and its adjustments to other bodies in their environment, taking into account object manipulations and body movements within the environment (Goodwin 2000). Recently, the consideration of mobility in interaction, comprising walking, driving and flying, has insisted on the importance of considering the entire body (Haddington, Mondada & Nevile 2013). What emerges from these studies is the necessity to go beyond the study of single ‘modalities’ coordinated with talk and to take into consideration the broader embodied and environmentally situated organization of activities (Goodwin 2012; Streeck, Goodwin & LeBaron 2011).
4The multimodal organization of turns, sequences and actions
The study of multimodality in conversation analysis elaborates on and contributes to the general literature about language and embodiment in social practices; it its also based on a specific analytic perspective, which puts in the foreground the ideas of action, temporality and indexicality. This praxeological conception of language and the body in interaction has developed into studies about how turns, sequences and actions are formatted and implemented thanks to multimodal resources.
Conversation analysis deals with the methodic way in which participants organize social interaction. This use of the notion of ‘method’ comes from ethnomethodology (Garfinkel 1967), and refers to the fact that participants build the intelligibility of their action in an orderly (that is, ‘methodic’) and publicly recognizable way (that is, ‘accountable’), which is both systematic and indexical, both transcending context and taking into account the diversity of contexts (both context-free and context-shaped, Heritage 1984). As a starting observation, Sacks, Schegloff and Jefferson (1974) demonstrated that participants interact by smoothly alternating turns at talk, minimizing both pauses and overlaps. This prompted a general interest in the methodic practices and resources through which this coordination happens in a finely tuned manner. Since the very first studies, a range of resources has been explored – concerning first linguistic resources, such as syntax, prosody and meaning, but then also embodied resources, such as gesture, gaze, head movements, nods, facial expressions and body postures.
Conversation analysis focuses on interactional order as it is achieved by participants at all levels of sequential organization. Participants engage step by step in the construction of their turns, which are formatted online, in an emergent way, taking into consideration the responses of co-participants (Goodwin 1979) and the contingencies of the interactional context. Turns unfold in a systematic way based on the projections and the normative expectations characterizing the organization of a sequence, first within the fundamental structure of the adjacency pair, constituted by a first pair part making relevant and expectable a second pair part, as in a question/answer sequence (Schegloff & Sacks 1973), and second within possible pre-sequences and post-expansions which complexify it (Schegloff 2007). Participants mobilize all the resources at hand for the intelligible, mutually accountable organization of the sequentiality of interaction; consequently, all the levels of sequential analysis have been explored as they are formatted and implemented not only by talk but also by a range of embodied resources.
Turn construction and turn-taking basically rely on multimodal resources for the organization of recognizable unit completions as well as transition-relevance places (Lerner 2003). Speakers can display their imminent self-selection by a range of multimodal resources used as ‘turn-entry devices’ – going from the ‘[a]-face’, displaying that the speaker is about talk, to the palm-up gesture (Streeck & Hartge 1992) and from the pointing gesture (Mondada 2007b) to other complex embodied manifestations (Schmitt 2005) – projecting that they are about to speak. Speakers also multimodally display the completion of turn-constructional units (TCUs) as well as turns by projecting completion thanks to the trajectory of gesture or its retraction (Mondada 2007b, 2015) used as ‘turn-exit devices’. More generally, Schegloff (1984: 267) suggests that the pre-positioning of gesture is a way of creating a ‘projection space’ within an on-going utterance. Turns can also be expanded, not only by adding syntactically fitted materials but also with gesture (Goodwin 1979, 1981) – for example, with gesture achieving the collaborative construction of turns, which is similar to the collaborative construction of utterances (Bolden 2003; Hayashi 2005; Iwasaki 2009).
Not only turn construction but also sequence organization rely on multimodal resources as demonstrated by studies on assessments (Goodwin & Goodwin 1987; Lindström & Mondada 2009), word searches (Goodwin & Goodwin 1986; Hayashi 2005), repair (de Fornel 1990–91; Greiffenagen & Watson 2007) – to give just a few examples.
Opening and closing sequences, as well as transition sequences from one activity to another, are typically organized in an embodied way. The opening of an interaction is achieved not only by the first words spoken or by the response to a summons, but, even before participants begin to speak, by an adjustment and arrangement of their bodies in the material environment in such a way that an ‘F-formation’ (Kendon 1990) adequate for the imminent activity is constituted. This in turn prompts the participants to progressively organize and assemble their bodies within the local environment, building the relevant ‘interactional space’ of the encounter (Hausendorf, Mondada & Schmitt 2012; Mondada, 2007a, 2009). They can even be engaged in different courses of action – or in multi-activity – as displayed for example by ‘body-torqued’ postures (Schegloff 1998) in which the upper part of the body is oriented towards a particular participation framework and the lower part to other relevant features. On the other hand, towards the closing of an encounter, the interactional space dissolves (De Stefani 2010; Robinson 2001). The importance of bodily movements in transitions between one episode of an interaction and another, often concomitant with the manipulation of objects and artefacts, also displays the embodied orientation of participants towards the organization of the interaction (Heath 1986; Modaff 2003; Mondada 2015).
5Materiality: objects, documents, technologies
Gestures are not meaningful per se in isolation but only in context and more precisely are ‘environmentally coupled’ (Goodwin 2007b). For instance, Goodwin (2009) shows that in order to analyse the turn, ‘She sold me this. But she didn’t sell me this (0.2) or tha:t’, one has to take into account ‘the integrated use of language, the body and objects in the world’ (107). Actually, the speaker of this utterance is holding a jar in his hand, which has been bought on the Internet. The missing part of the object is made visible by rotating the hand at the bottom of the jar, and the words are made meaningful by the articulation among talk, gesture and the object. As Goodwin notes, gestures coupled to phenomena in the environment are pervasive in many settings. Even more radically, LeBaron and Streeck (2000) consider that ‘hands-on interactions with things’ and tactile manipulations of objects constitute the experiential grounding of more abstract and symbolic conversational gesture; symbols embody experiences that have emerged in situated action (136) and ‘hands learn how to handle things before they learn how to gesticulate’ (137; see Streeck 2009).
Besides ordinary objects, video analyses in ethnomethodology and conversation analysis have paid special attention to particular artifacts such as texts and visualizations, which are both types of objects that can be handled, grasped and manipulated with the hands, and semiotic objects that can be read. In medical settings, the way doctors turn to such objects during consultation (DiMatteo et al. 2003; Heath 1986; Robinson 1998) or produce texts and files (Heath & Luff 1996) has been documented. Likewise, the reflexive constitution of artifacts and inscriptions by the way in which they are locally mobilized within work activities (Mondada 2006; Suchman 2000) and during scientific activities (Ochs, Gonzales & Jacoby 1996; Roth & Lawless 2002) has been studied. Goodwin’s study of the use of the Munsell chart by archaeologists (1999) details this complex web of multimodal resources in an exemplary way. He studies archaeologists excavating soil and examining its colour by holding a coding form to be filled with soil description and a Munsell chart allowing the comparison between different shades of colour. The mobilization of the chart is done in an ordered way, aligned with other movements such as body arrangements, participants’ gaze orientations, their pointing gestures, the holding of a trowel, as well as with talk – all these resources making it possible to compare the colours of the Munsell chart with the colour of the soil sample. ‘Seeing’ colour with the Munsell chart is not the automatic result of a procedure; rather, it is a situated achievement needing the prior alignment of the local action space and, thus, requiring time.
Other forms of materialities have been studied in relation to social interactions in environments in which technological tools are used, manipulated and gazed at. Workplace studies have shown the pervasive configuring role of materiality and artifacts in professional activities such as in airport control rooms (Goodwin & Goodwin 1996: 62; Suchman 1993), underground stations (Heath & Luff 2000), surgical theatres (Koschmann et al. 2007, Mondada 2003, 2011b), television studios (Broth 2008, 2009), emergency call and call centres (Fele 2008; Mondada 2008; Whalen 1995; Whalen & Zimmerman 1987), pilot’s cockpits (Nevile 2004), etc.
In summary, attention to objects permits a detailed investigation not only of the manual actions of the hands but also of involvements of the entire body, as well as of the way in which those actions are anchored within the spatial environment.
6Spatiality: the establishment and transformation of interactional spaces
The focus on bodies and objects entails a renewed interest in the environment and in the way in which embodied conducts adapt, exploit and transform the features of the material surroundings of an action. In this sense, space refers not just to the pre-existing environment but also to a configuration created by the embodied disposition, orientation and arrangement of the participants within an interaction.
Early on, Goffman (1963, 1964) showed that body arrangements in space create temporary territories with changing boundaries. These territories are recognized by participants involved in an encounter and also by bystanders. Body positions delimit a temporary ‘ecological huddle’ (Goffman 1964), which materializes the ‘situated activity system’. These arrangements constitute what Goffman (1963) calls ‘focused gatherings’, which are defined by mutual orientation and shared attention, as displayed by body positions, postures, gazes and addressed gestures. This interest in temporary and labile territories and in their effectiveness is shared by Scheflen (1972). On the basis of encounters video-taped in private and public settings, he describes how a group talking together draws a claimed territory, which is observed and respected by others outside the circle. Kendon (1977, 1990) conceptualizes this territory by using the notion of ‘F-formation’, referring to how different body positions and orientations build an arrangement favouring a common focus of attention and engagement in a joint activity.
In his work, Goodwin (2000, 2003, 2007b) insists on the mutual relationship between embodied actions and the material environment, defining what he calls a ‘contextual configuration’. If the analysis of talk has to take into consideration the embodied actions of participants, the study of gesture or body posture cannot be developed in isolation but has to describe the way in which the structure of an environment contributes to the organization of an interaction.
Drawing on these inspirations, Mondada (2007a, 2009, 2011a) proposes that ‘interactional space’ is constituted through the situated, mutually adjusted and changing arrangements of participants’ bodies within space. This produces a configuration relevant to the activity they are engaged in, their mutual attention and their common focus of attention, the objects they manipulate and the way in which they coordinate in joint action. This interactional space is constantly being established and transformed within an activity (De Stefani 2011; Hausendorf, Mondada & Schmitt 2012; LeBaron & Streeck 1997; Mondada 2009, 2011a). The dynamic transformation of interactional space is achieved by the bodily arrangements of participants constituting mobile configurations and mobile formations. This is even more the case with interactional spaces constituted through and within mobile activities such as walking, driving, biking, etc. Mobility further enlarges and dynamizes the vision of space in interaction.
7Mobility: bodies in movement
The focus on space has also prompted observations about how the entire body is put in motion in social interaction. Traditionally, much research has predominantly focused on interactions in static settings or within a local site, and less attention has been paid to interactions occurring in a mobile situation, either with bodies in motion, as in walking, or with participants moving in a car (Haddington, Keisanen & Nevile 2012) or even in an airplane (Nevile 2004).
Walking in interaction is an interesting case of mobility because it involves the entire body and the complex bodily arrangements of dynamic interactional spaces.
Early work on walking already describes ‘doing walking’ as a methodic practice and a concerted accomplishment (Ryave & Schenkein 1974: 265). Members achieve walking together, being recognized both as a ‘vehicular unit’ (Goffman 1971: 8) and as ‘withs’ (Goffman 1971: 19). In walking together, participants organize their concerted action both within a group – by maintaining proximity and pace, speeding up and slowing down, managing turns and stopping together (see De Stefani 2011; Haddington, Mondada & Nevile 2013) – and with respect to other passers-by, – while navigating within a crowd, avoiding collisions and adjusting to the trajectory of others and even while making accountable the interruption of their trajectory (Watson 2005). Two mobile units can also converge, for example when various ‘withs’ meet and merge, thereby constituting one unique interactional space (Mondada 2009). Conversely, people can also display that they are not with, exhibiting civil inattention and minimizing the effects of co-presence (Goffman 1971; Sudnow 1972). As noted by Ryave and Schenkein (1974), the fact that these challenges are resolved in unproblematic ways reveals ‘the nature of the work executed routinely by participant walkers’ (267).
Moreover, collective walking activities are organized by being oriented in a finely-tuned way to the organization of talk and even to the details of the emergent construction of turns and sequences; Relieu (1999) shows how turn-design is sensitive to the spatial ecology encountered by speakers talking and walking, and Mondada (2009) shows how the first turn of an encounter is finely designed with respect to the walking body of the co-participant. Mobile practices such as walking away (Broth & Mondada 2013) or running away (Depperman, Schmitt & Mondada 2010) orient to transition-relevance points and to transitions from one activity to the other; additionally, they contribute to the achievement of these transitions and to their accountability in a publicly visible way to which all co-participants can align – or eventually disalign.
8An empirical case: the multimodal organization of reference
In order to discuss empirical issues related to the dynamic establishment of multimodal gestalts (Mondada 2015), we turn now to an example, which shows how a participant introduces a new referent in his talk while the other co-participants are still focused on a previous object. The fragment is extracted from a guided visit lead by a gardener, LUC, who describes and explains the rich fauna and flora of a famous garden to YAN, ELIse, and JEAn. Such a context and activity involves in a crucial way a) the gesture of the main participant pointing around him, b) his talk, c) the mobility of the participants who are walking along the footpath, d) their distribution and mutual positioning designing a dynamic interactional space and e) their gaze and joint attention since they are bodily and visually oriented towards the rich details of the material environment.
We join the fragment as Luc begins to point to a dead branch where woodpeckers have produced visible scars – while the other participants are still looking at the previous object pointed at, a tree supported by a stick.
The excerpt has been video recorded by a mobile camera held by a person walking backwards along the footpath as the group walks forwards.
The fragment is transcribed both for verbal actions (numbered lines, followed by a translation, preceded by the identification of the speaker in capitals) and embodied actions (in the next line, with the identification of the participants in small letters). Embodied actions are synchronized with talk by a series of symbolic landmarks (one per participant), permitting the reader to reconstruct the simultaneous and successive temporality of the multimodal resources.
Extract 1 (3/15.25 piverts)
While Jean is still commenting about the supported tree (1) and Elise is responding to him (2), Luc leaves the footpath and approaches another tree, directing his body towards it and beginning to point at it. He initiates the next action, focusing on the next object to be looked at and described, but his co-participants are still engaged with the previous object, looking away (Jean walks forward on the path, Elise looks at him and Yan looks at the tree – see Figure 1). Both the transcription and the screen shot make available a rich array of multimodal details showing diverging orientations of the participants.
The practical problem with which Luc is confronted is how, in this circumstance, to point and show them the detail he is beginning to refer to. The practical solution he finds consists of both delaying the progressivity of his talk and of establishing the joint attention of the participants before continuing with his description.
We can notice that, in line 3, Luc begins his turn with ‘oui, mais’/’yes, but’; he both treats what Jean has just said (1) and operates a disjunction with it, introducing an alternative action. Moreover, he first utters the verb ‘r’gardez’/‘look’ alone, then the same verb with a definite article, which projects a name but is not followed by it. The unfinished sentence is overlapped by Elise who is still orienting to Jean’s comment. Jean repeats the verb again (5), followed this time by the article and the noun – referring to the object he is introducing (actually invisible but represented by the scars left by the woodpeckers on the pointed at branch).
During this new re-start of Luc’s turn, different embodied conducts are deployed by the participants; Luc has reached the tree and stops pointing, turning to the co-participants; the co-participants now look at him, Elise being already reoriented towards him while Jean and Yan still walk on the footpath. Yan has just asked a question about the previous tree and turns to Luc as the expert person able to respond; he sees Jean pointing at the tree and reorients his gaze at the pointed at place.
In line 8, Luc repeats the entire utterance. Contrary to line 5, which was very expressive, the utterance is produced with a flat voice; the utterance was attracting the gaze of the co-participants line 5, working as an attention-getting device, whereas line 8 it introduces the referent to be seen. This turn is now responded to not only by the establishment of a new interactional space around him (Figure 2), but also by a change-of-state token produced by Yan (10). At that point, Luc continues predicating something about the woodpeckers and goes on with a story about them (11 ff.). Elise produces an acknowledgement too (12).
This excerpt shows how a multiplicity of embodied resources are mobilized to achieve the introduction of the new focus of attention; on the side of the speaker, an array of linguistic resources are used. These include the verb ‘to look’ in the imperative, the projective power of the article, the delay of the noun, the pointing gesture and the entire body not only approaching the object but also leaning towards it, as a kind of bodily pointer, not to forget the gaze monitoring the action of the co-participants. On the side of the co-participants, an emerging responsive action occurs in form of a change of direction of the gaze a reorientation of the bodies stopping to walk and turning to the object, together with a verbal response acknowledging what Luc is doing. Only when this complex web of multimodal transformations has been achieved can Luc continue with the progressivity of his talk.
This example confirms previous work on the embodied conditions of the use of deictics (Goodwin 2003; Hindmarsh & Heath 2000; Mondada 2007a). Prior to the use of referential expressions, a speaker engages in intensive interactional work in order not only to get the relevant attention of co-participants but also to (re)arrange their entire bodies in such a way that a deictic action (achieved through linguistic and gestural resources) lies within the focus of attention of the participants. As Goodwin (2003) shows, pointing and talking are formatted together by taking into consideration the surrounding space, the activity in which the participants are engaged and the participants’ mutual orientation. In the study using the example of archaeologists excavating soil, Goodwin (2000) shows how participants actively constitute a visual field that has to be scrutinized, parsed and understood together by the co-participants in order to find out where the speaker is pointing. The archaeologists juxtapose language, gesture, tools (such as trowels) and graphic fields (such as maps) on a domain of scrutiny, which is surrounding them but is also being delimitated by the very act of referring to it. In this sense, gestures are environmentally coupled (Goodwin 2007b) and not used as a separated resource coming from the exterior world into a pre-existing context; the domain of scrutiny is transformed and reorganized by the very action of pointing done within the current task.
As Hindmarsh and Heath (2000) show, these gestures and body movements and amplifying them are realized in a way that is recipient-designed, that is, participants indicate and even display referents for co-participants at the relevant moment when an object is visible to them. Pointing gestures are ‘produced and timed with respect of the activities of the co-participants, such that they are in a position to be able to see the pointing gesture in the course of its production’ (Hindmarsh and Heath 2000: 1868). Thus, the organization of the gesture and the body of a speaker is adjusted to the recipient in order to guide him or her in the material environment towards the referent. Since recipients display their understanding and grasp of an action going on, speakers adjust to the production of these expressions or to their absence or delay.
This mutual orientation involves not only talk and gesture but also the entire body, gazing on and bending towards the object and, more radically, actively rearranging the surrounding environment. Mondada (2007a) shows how speakers, prior to the production of a deictic, dispose their bodies within space, reposition objects within space and even restructure the environment. The deictic and the pointing gesture are produced only after participants have organized the disposition of their spatial context. Thus, deictic words and gestures are not merely adapting to a pre-existing and immutable context; they are part of an action that actively renews and changes the context, rearranging the interactional space in the most appropriate way for the pointing gesture to take place. In these cases, the emergent and progressive temporality of talk and action is suspended, delayed or postponed until the conditions for joint attention or a common focus of attention are fulfilled.
The emergent organization of talk and action concerns not only gesture and gaze but also the moving body, the surrounding space and the material environment. What emerges from these contributions is the necessity to go beyond the study of single ‘modalities’ coordinated with talk and to take into consideration the broader embodied and environmentally situated organization of activities (Streeck, Goodwin & LeBaron 2011).
9Conclusion: challenges
Multimodal interaction opens up an extremely rich field of investigation, expanding prior knowledge about social interaction. A wider notion of the multimodal resources mobilized by participants for building their accountable actions includes language, gesture, gaze, facial expression, body posture, body movement, such as walking, and embodied manipulation of artifacts. This enlarged vision opens up various challenges, both methodological and theoretical.
Methodologically, the study of relevant details concerning the entire body challenges the way in which social action is documented, first through video recordings of naturally occurring interactions in their ordinary social settings and second through transcripts and other forms of annotation. The documentation of participants engaged in mobile activities within complex settings, involving not only their bodies but also various material and spatial environmental details, requires video recordings and video technologies that are relevantly adjusted to the activities observed. The representation of complex conduct involving a variety of multimodal resources also challenges traditional, more linear, transcripts and requires more and more sophisticated annotation and alignment tools (like ELAN, ANVIL or CLAN).
Analytically, this rich documentation shows the complexity of the reconstruction of the moment-by-moment temporality of an emerging interaction. Multimodality is characterized by multiple temporalities, both simultaneous and successive, and by multiple sequentialities operating at the same time. More broadly, complex multimodal gestalts require a study of the social organization of language and embodied conducts by taking into consideration both their systematicity and the specificity of their ecology – as well as participants’ orientation towards these multiple details unfolding in real time.
Transcript conventions
Talk has been transcribed according to conventions developed by Gail Jefferson (2004).
An indicative translation is provided line per line, in italics. It aims at making the original understandable for the reader.
Multimodal details have been transcribed according to the following conventions (see Mondada 2007b https://franz.unibas.ch/fileadmin/franz/user_upload/redaktion/Mondada_conv_multimodality.pdf):
| * * | delimit descriptions of one speaker’s actions. |
| + + | delimit descriptions of another speaker’s actions. |
| *---> | action described continues across subsequent lines. |
| *--->> | action described continues until and after excerpt’s end. |
| ---->* | action described continues until the same symbol is reached. |
| >>-- | action described begins before the excerpt’s beginning. |
| .... | action’s preparation. |
| ,,,,, | action’s retraction. |
| luc | participant doing the action is identified in small characters. |
| fig | figure; screen shot. |
| # | indicates the exact moment at which the screen shot has been recorded. |
References
Bavelas, Janet Beavin & Nicole Chovil. 2000. Visible acts of meaning. An integrated message model of language in face-to-face dialogue. Journal of Language and Social Psychology 19(2). 163–193.
Bavelas, Janet Beavin, Nicole Chovil, Douglas A. Lawrie & Allan Wade. 1992. Interactive Gesture. Discourse Processes 15. 469–489.
Birdwhistell, Ray L. 1970. Kinesics and Context. Philadelphia: Univ. of Pennsylvania Press.
Bohle, Ulrike. 2001. Das Wort ergreifen das Wort übergeben. Berlin: Weidler Buchverlag.
Bolden, Galina B. 2003. Multiple modalities in collaborative turn sequences. Gesture 3(2). 187–212.
Broth, Mathias. 2008. The studio interaction as a contextual resource for TV-production. Journal of Pragmatics 40(5). 904–926.
Broth, Mathias. 2009. Seeing through screens, hearing through speakers: Managing distant studio space in television control room interaction. Journal of Pragmatics 41. 1998–2016.
Broth, Mathias & Lorenza Mondada. 2013. Walking away: the embodied achievement of activity closings in mobile interaction. Journal of Pragmatics. 47(1). 41–58.
Clark, Herbert. 1996. Using Language. Cambridge: Cambridge University Press.
Condon, William S. 1971. Speech and Body Motion Synchrony of the Speaker-Hearer. In David L. Horton & James J. Jenkins (eds.), Perception of Language, 150–173. Columbus: Merrill.
De Stefani, Elwys. 2010. Reference as an interactively and multimodally accomplished practice : Organizing Spatial Reorientation in Guided Tours. In Massimo Pettorino, Antonella Giannini, Isabella Chiari & Francesca M. Dovetto (eds.), Spoken Communication, 137–170. Newcastle upon Tyne: Cambridge Scholars Publishing.
De Stefani, Elwys. 2011. Ah petta ecco, io prendo questi che mi piacciono’. Agire come coppia al supermercato. Un approccio conversazionale e multimodale allo studio dei processi decisionali. Roma: Aracne.
Deppermann, Arnulf, Reinhold Schmitt & Lorenza Mondada. 2010. Agenda and emergence: Contingent and planned activities in a meeting. Journal of Pragmatics 42. 1700–1712.
DiMatteo, M. Robin, Jeffrey D. Robinson, John Heritage, Melissa Tabbarah & Sarah A. Fox. 2003. Correspondence among patients’ self-reports, chart records, and audio/videotapes of medical visits. Health Communication 15. 393–413.
Drew, Paul & John Heritage (eds.). 1992. Talk at Work. Cambridge: Cambridge University Press.
Fele, Giolo. 2008. The collaborative production of responses and dispatching on the radio: Video analysis in a medical emergency call centre. Forum: Qualitative Social Research 9(3). art. 40.
Fornel, Michel. 1990–91. De la pertinence du geste dans les séquences de réparation et d’interruption. In Bernard Conein, M. d. Fornel & Louis Quéré (eds.), Les formes de la conversation, Vol. 2, 119–154. Paris: CNET.
Garfinkel, Harold. 1967. Studies in Ethnomethodology. Englewood Cliffs, N.J.: Prentice-Hall.
Goffman, Erving. 1963. Behavior in Public Places: Notes on the Social Organization of Gathering. New York: Free Press.
Goffman, Erving. 1964. The neglected situation. American Anthropologist 66(6). 133–136.
Goffman, Erving. 1971. Relations in Public: Microstudies of the Public Order. New York: Harper and Row.
Goodwin, Charles. 2012. The co-operative, transformative organization of human action and knowledge. Journal of Pragmatics 46(1). 8–23.
Goodwin, Charles. 1979. The interactive construction of a sentence in natural conversation. In George Psathas (ed.), Everyday Language: Studies in Ethnomethodology, 97–121. New York: Irvington Publishers.
Goodwin, Charles. 1981. Conversational Organization: Interaction Between Speakers and Hearers. New York: Academic Press.
Goodwin, Charles. 1999. Practices of color classification. Mind, Culture and Activity 7(1–2). 62–82.
Goodwin, Charles. 2000. Action and embodiment within situated human interaction. Journal of Pragmatics 32. 1489–1522.
Goodwin, Charles. 2003. Pointing as Situated Practice. In Sotaro Kita (ed.), Pointing: Where Language, Culture and Cognition Meet, 217–241. Hillsdale: L. Erlbaum.
Goodwin, Charles. 2007a. Participation, stance and affect in the organization of activities. Discourse and Society 18(1). 53–73.
Goodwin, Charles. 2007b. Environmentally Coupled Gestures. In Susan Duncan, Justine Cassell & Elena T. Levy (eds.), Gesture and the Dynamic Dimensions of Language, 195–212. Amsterdam, Philadelphia: John Benjamins.
Goodwin, Charles. 2009. Things, Bodies, and Language. In Bruce Fraser & Ken Turner (eds.), Language in Life, and a Life in Language: Jacob Mey – A Festschrift, 106–109. Bingley (UK): Emerald.
Goodwin, Charles & Marjorie H. Goodwin. 1987. Concurrent operations on talk: Notes on the interactive organization of assessments. Pragmatics 1(1). 1–55.
Goodwin, Charles & Marjorie H. Goodwin. 1996. Seeing as a situated activity: Formulating planes. In Yrjo Engeström & David Middleton (eds.), Cognition and Communication at Work, 61–95. Cambridge: Cambridge University Press.
Goodwin, Marjorie H. & Charles Goodwin. 1986. Gesture and coparticipation in the activity of searching for a word. Semiotica 62(1–2). 51–75.
Greiffenhagen, Christian & Rod Watson. 2007. Visual Repairables: analyzing the work of repair in human-computer interaction. Visual Communication 8(1). 65–90.
Haddington, Pentti, Maurice Nevile & Tiina Keisanen (eds.). 2012. Meaning in Motion: Interaction in Cars. [Special issue]. Semiotica 191.
Haddington, Pentti, Lorenza Mondada & Maurice Nevile (eds.). 2013. Mobility and interaction. Berlin: De Gruyter.
Hausendorf, Heiko, Lorenza Mondada & Reinhold Schmitt (eds.). 2012. Raum als interaktive Resource. Tübingen: Narr.
Hayashi, Makoto. 2005. Joint turn construction through language and the body. Semiotica 156(1/4). 21–53.
Heath, Christian. 1986. Body Movement and Speech in Medical Interaction. Cambridge: Cambridge University Press.
Heath, Christian. 1989. Pain talk: the expression of suffering in the medical consultation. Social Psychology Quarterly 52(2). 113–125.
Heath, Christian & Paul Luff. 1996. Documents and professional practices: ‘bad’ organisational reasons for ‘good’ clinical records. Proceedings of the Conference on Computer Supported Cooperative Work. 354–363.
Heath, Christian & Paul Luff. 2000. Technology in Action. Cambridge: Cambridge University Press.
Heath, Christian, Jon Hindmarsh & Paul Luff. 2010. Video in Qualitative Research. London: Sage.
Heritage, John C. 1984. Garfinkel and Ethnomethodology. Cambridge & New York: Polity Press.
Hindmarsh, John & Christian Heath. 2000. Embodied reference: A study of deixis in workplace interaction. Journal of Pragmatics 32. 1855–1878.
Iwasaki, Shoichi. 2009. Initiating interactive turn spaces in Japanese conversation: Local projection and collaborative action. Discourse Processes 46. 226–246.
Jefferson, Gail. 2004. A sketch of some orderly aspects of overlap in natural conversation. In Gene Lerner (ed.), Conversation Analysis: Studies from the first generation, Amsterdam: Benjamins.
Jefferson, Gail. 2004. A sketch of some orderly aspects of overlap in natural conversation. In Gene Lerner (ed.), Conversation Analysis: Studies from the first generation, 13–32 Amsterdam: Benjamins.
Kendon, Adam. 1967. Some Functions of Gaze-Direction in Social Interaction. Acta Psychologica 26. 22–63.
Kendon, Adam. 1970. Movement Coordination in Social Interaction. Acta Psychologica 29. 100–125.
Kendon, Adam. 1980. Gesture and Speech: Two Aspects of the Process of Utterance. In Mary R. Key (ed.), Nonverbal Communication and Language, 207–277. The Hague: Mouton.
Kendon, Adam. 1990. Conducting Interaction: Patterns of Behavior in Focused Encounters. Cambridge: Cambridge University Press.
Kendon, Adam. 2004. Gesture. Visible Action as Utterance. Cambridge: Cambridge University Press.
Koschmann, Timothy, Curtis LeBaron, Charles Goodwin, Alan Zemel & Gary Dunnington. 2007. Formulating the triangle of doom. Gesture 7(1). 97–118.
Lebaron, Curtis D. & Jürgen Streeck. 1997. Built space and the interactional framing of experience during a murder interrogation. Human Studies 20. 1–25.
Lebaron, Curtis D. & Jürgen Streeck. 2000. Gestures, knowledge and the world. In David McNeill (ed.), Language and Gesture, 118–138. Cambridge: Cambridge University Press.
Lerner, Gene H. 2003. Selecting next speaker: The context-sensitive operation of a context-free organization. Language in Society 32. 177–201.
McNeill, David. 1985. So you think gestures are nonverbal? Psychology Review 92(3). 350–371.
McNeill, David. 1992. Hand and Mind: What Gestures Reveal About Thought. Chicago: University of Chicago Press.
McQuown, Norman A. (ed.). 1971. The Natural History of an Interview. Chicago: University of Chicago.
Modaff, Daniel P. 2003. Body Movement in the Transition From Opening to Task in Doctor-Patient Interviews. In Baron, Curtis D., Jenny Mandelbaum & Phillip J. Glenn (eds.), Studies in Language and Social Interaction: In honor of Robert Hopper, 411–422. Mahwah: Erlbaum.
Mondada, Lorenza. 2003. Working with video: how surgeons produce video records of their actions. Visual Studies 18(1). 58–72.
Mondada, Lorenza. 2006. Video recording as the reflexive preservation-configuration of phenomenal features for analysis. In Hubert Knoblauch, Bernt Schnettler, Jürgen Raab & Hans-Georg Soeffner (eds.), Video Analysis. Bern: Lang.
Mondada, Lorenza. 2007a. Interaktionsraum und Koordinierung. In Arnulf Depperman & Reinhold Schmitt (eds.), Koordination. Analysen zur multimodalen Interaktion, 55–94. Tübingen: Narr.
Mondada, Lorenza. 2007b. Multimodal resources for turn-taking: Pointing and the emergence of possible next speakers. Discourse Studies 9(2). 195–226.
Mondada, Lorenza. 2008. Using video for a sequential and multimodal analysis of social interaction: Videotaping institutional telephone calls. FQS (Forum: Qualitative Sozialforschung 9(3). www.qualitative-research.net (accessed 18 March 2015).
Mondada, Lorenza. 2009. Emergent focused interactions in public places: A systematic analysis of the multimodal achievement of a common interactional space. Journal of Pragmatics 41. 1977–1997.
Mondada, Lorenza. 2011a. The interactional production of multiple spatialities within a participatory democracy meeting. Social Semiotics 21(2). 283–308.
Mondada, Lorenza. 2011b. The organization of concurrent courses of action in surgical demonstrations. In Jürgen Streeck, Charles Goodwin & Curtis LeBaro (eds.), Embodied Interaction, Language and Body in the Material World, 207–226. Cambridge: Cambridge University Press.
Mondada, Lorenza. 2012. The Conversation Analytic Approach to Data Collection. In Jack Sidnell & Tanya Stivers (eds.), Handbook of Conversation Analysis, 32–56. London: Blackwell-Wiley.
Mondada, Lorenza. 2014. The local constitution of multimodal resources for social interaction. Journal of Pragmatics 65. 137–156.
Mondada, Lorenza. 2015. Multimodal completions. In : Deppermann, A., Günthner, S., Temporality in Interaction. Amsterdam: Benjamins, 267–307
Mondada, Lorenza & Reinhold Schmitt (eds.). 2010. Situationseröffnungen: Zur multimodalen Herstellung fokussierter Interaktion. Tübingen: Narr.
Müller, Cornelia. 1998. Redebegleitende Gesten. Kulturgeschichte – Theorie – Sprachvergleich. Berlin: Berlin Verlag.
Nevile, Maurice. 2004. Beyond the Black Box: Talk-in-interaction in the Airline Cockpit. Aldershot: Ashgate.
Ochs, Elinor, Patrick Gonzales & Sally Jacoby. 1996. When I come down I’m in the domain state: Grammar and graphic representation in the interpretive activity of physicists. In Elinor Ochs, Emanuel A. Schegloff & Sandra A. Thompson (eds.), Interaction and Grammar, 328–369. Cambridge: Cambridge University Press.
Peräkylä, Anssi & Johanna Ruusuvuori. 2006. Facial expression in an assessment. In Hubert Knoblauch, Bernt Schnettler, Jürgen Raab & Hans-Georg Soeffner (eds.), Video analysis: methodology and methods, 127–142. Frankfurt: Peter Lang.
Relieu, Marc. 1999. Parler en marchant. Pour une écologie dynamique des échanges de paroles. Langage et Société.89. 37–68.
Robinson, Derek. 1998. Getting down to business: Talk, gaze, and body orientation during openings in doctor-patient consultation. Human Communication Research 25. 98–124.
Robinson, Derek. 2001. Closing medical encounters: two physician practices and their implications for the expression of patients’ unstated concern. Social Science & Medecine 53. 639–656.
Roth, Wolff-Michael & Daniel V. Lawless. 2002. When up is down and down is up: body orientation, proximity, and gestures as resources. Language in Society 31. 1–28.
Ryave, A. Lincoln & James N. Schenkein. 1974. Notes on the art of walking. In Roy Turner (ed.), Ethnomethodology, 265–274. Harmondsworth: Penguin.
Sacks, Harvey & Emanuel A. Schegloff. 2002. Home position. Gesture 2(2). 133–146.
Sacks, Harvey, Emanuel A. Schegloff & Gail Jefferson. 1974. A simplest systematics for the organization of turn-taking for conversation. Language 50. 696–735.
Scheflen, Albert. 1972. Body Language and Social Order: Communications as Behavioral Control. Englewood Cliffs: Prentice Hall.
Schegloff, Emanuel A. 1984. On Some Gestures’ Relation to Talk. In Maxwell Atkinson & John Heritage (eds.), Structures of Social Action, 266–296. Cambridge: Cambridge University Press.
Schegloff, Emanuel A. 1998. Body torque. Social Research 65(3). 535–586.
Schegloff, Emanuel A. 2007. Sequence Organization in Interaction: A Primer in Conversation Analysis, Vol. 1. Cambridge: Cambridge University Press.
Schegloff, Emanuel A. & Harvey Sack. 1973. Opening up closings. Semiotica 8. 289–327.
Schegloff, Emanuel A., Gail Jefferson & Harvey Sacks. 1977. The Preference for Self-Correction in the Organization of Repair in Conversation. Language 53. 361–382.
Schmitt, Raymond. 2005. Zur multimodalen Struktur von turn-taking. Gesprächsforschung 6. 17–61.
Seyfeddinipur, Mandana & Marianne Gullberg (eds.). 2014. From Gesture in Conversation to Visible Utterance in Action. Amsterdam: Benjamins.
Stivers, Tanya. 2008. Stance, alignment and affiliation during storytelling: When nodding is a token of affiliation. Research on Language and Social Interaction 41(1). 31–57.
Streeck, Jürgen. 1993. Gesture as communication I: its coordination with gaze and speech. Communication Monographs 60. 275–299.
Streeck, Jürgen. 2009. Gesturecraft: The Manufacture of Understanding. Amsterdam: Benjamins.
Streeck, Jürgen & Ulrike Hartge. 1992. Previews: gestures at the transition place. In Peter Auer & Aldo Di Luzio (eds.), The Contextualization of Language, 135–157. Amsterdam: Benjamins.
Streeck, Jürgen, Charles Goodwin & Curtis LeBaron (eds.). 2011. Embodied Interaction, Language and Body in the Material World. Cambridge: Cambridge University Press.
Suchman, Lucy. 1993. Technologies of accountability: Of lizards and airplanes. In Graham Button (ed.), Technology in Working Order: Studies of Work, Interaction and Technology, 113–126. London: Routledge.
Suchman, Lucy. 2000. Making a case: “knowledge” and “routine” work in document production. In Paul Luff, Jon Hindmarsh & Christian Heath (eds.), Workplace Studies. Cambridge: Cambridge University Press.
Sudnow, David. 1972. Temporal parameters of interpersonal observation. In David Sudnow (ed.), Studies in Social Interaction, 259–279. New York: Free Press.
Watson, Reg. 2005. The visibility arrangements of public space: conceptual resources and methodological issues in analysing pedestrian movements. Communication & Cognition, 38(1–2). 201–227.
Whalen, Jack. 1995. Expert systems versus systems for experts: Computer-aided dispatch as a support system in real-world environments. In Peter Thomas (ed.), The Social and Interactional Dimensions of Human-Computer Interfaces, 161–183. Cambridge: Cambridge University Press.
Whalen, Marilyn R. & Don H. Zimmerman. 1987. Sequential and Institutional Contexts in Calls for Help. Social Psychology Quarterly 50. 172–185.