Interaction (and its variants, interactive and interactivity) has seized public consciousness with the advent and rapid proliferation of the Internet, digital media, broadband networking, mobile devices, video games, and other electronic forms. Indeed, “interactive” has become a loaded marketing buzzword, synonymous with sleek, glamorous hi-tech lifestyles of the “digiterati.” If something is labeled as interactive, it is instantly regarded as modern and, supposedly, very marketable. However, as a fundamental concept of human living, interaction has nothing inherently to do with computers. Instead, it is a technologically agnostic concept, with a wide range of applicability: from reading a book, cooking dinner with friends, painting a picture, to drafting budgets and timelines—all which may be augmented with tools and technologies. Indeed, this concept lies at the core of many situations people face in daily life, from science and business to religion and family.

In its purest form interaction refers to a dynamic relationship between reciprocating entities at varying types and degrees of influence: people, environment, natural forces, and spiritual/cultural ideals. Obviously, interaction by itself has little value and thus needs a framework to enable a useful discussion. User-centered design (UCD) within the domain of human-computer interaction (HCI) provides this organizing element to focus the discussion on design-oriented phenomena. This domain concerns itself primarily with the design of user interfaces, web-based media, mobile devices, and software systems—thus, computer mediated experiences. Design, in this case, means the conception, planning, and making of “the artificial” (products, services, systems, environments) that serve individual and collective human goals. It is a situated activity, dependent upon the circumstances of use (as well as the conditions of product development). It is also a deeply human enterprise, contingent upon personal skills in imagination, empathy, synthetic thinking, and visual communication. The following professional insights further amplify this notion of Design into the realm of interactions:

So, while we may think of popular expressions of Interaction Design like buttons, links, icons, menus, tabs, on a computer screen, we should consider Herb Simon's scholarly account that “everyone designs who devises courses of action aimed at changing current situations into preferred ones.”⁷⁶

Language is yet another ambiguous concept for which varieties of theories flourish in the realms of linguistics, cognitive psychology, and sociology. We should note two of many influential contributors to the philosophy of language: Ferdinand Saussure and Ludwig Wittgenstein.

Saussure was a late 19^th—early 20^th century Swiss linguist; his thinking focused on the formal systemization of signs (for example, icons and buttons). He approached linguistics as a branch of a broader science of signs, which he labeled semiology (now semiotics). This theory involved signs as the basic unit of a specific language, which he considered as a comprehensive system of signs. As we'll see later, this notion has applicability to design in creating visual iconography systems for graphical interfaces and wayfinding.

Ludwig Wittgenstein was a 20^th century Austrian philosopher of language and mind, who helped evolve linguistic theory. Wittgenstein initially prescribed language as pictorial representation of relationships, but later repudiated this, instead advancing the modern notion that the meaning of a word is found in its use in the context of a “language-game.” Thus words function and receive their meaning within a context, rather than as atomic, logically predetermined facts or pictures of meaning. For Wittgenstein, the construct of a language game was an “orientation toward action and experience that provided the context for determining meaning.”⁷⁷ Language is perceived as a human activity, dependent upon the setting of the word used by humans. So, language becomes a shared linguistic practice, which sounds very appropriate to Interaction Design, implying relevant social value.

For our purpose of understanding Interaction Design, we will look at language in terms of situated human use and action. Indeed, we may collectively assume some basic notions at an intuitive level of common use. For example: Language generally refers to that which arises from the systematic integration of utterances, words, or phrases into a regulated pattern of expression, governed by rules for grammar and syntax. Language is a means to facilitate the exchange of information (i.e., communication) from senders to recipients. Ideas, thoughts, emotions, values may be delivered via language. Ultimately, language is about meaning, the creation and delivery of linguistic value (more about social and cultural later).

Language, however, is not the same as communication but a necessary precedent. From common observations of politics and advertising we can presume that language is a potent force influencing perceptions of reality. The sense impression of a word (i.e., connotation) given a cultural context and practical situation can render positive or negative affect of the word upon its reference, impelling certain actions, like voting for a ballot initiative or purchasing one brand of toothpaste over another. Without language there cannot be “actionable” communication that affects one's emotional and linguistic state (i.e., productive debate, etc.). Thus, language is used to identify, qualify, characterize, interpret, and color the phenomena of the world (our reality as we experience and share it). Language is a potent force for designers to comprehend in crafting effective interactions. Of course, a well-rounded review of language requires mention of basic concepts like framing, medium, metaphor, and, the creation of meaning itself.

Lakoff and Johnson captivated HCI audiences in the 1970s with their critical work entitled Metaphors We Live By, suggesting the primacy of metaphorical thinking in understanding the world's phenomena (i.e., material reality). George Lakoff is a professor of cognitive linguistics, which focuses on the relation between language and underlying mental processes of human cognition in ascertaining reality. According to Lakoff, metaphors, frames, and mappings are necessary interrelated aspects of language's utility in ordinary human correspondence.

Metaphors are “ways of interpreting our daily world with previously experienced and known relationships/associations to enhance meaning, and achieve shared understanding.”⁷⁸ They are linguistic constructs for seeing one thing in terms of another. But a metaphor is more than just poetic flourish for embellishment. The human conceptual cognitive system is “fundamentally metaphorical in nature.” Indeed metaphors are intrinsically connected to normal human thought patterns, how people conceive and perceive reality, particularly abstractions that require multiple metaphors for maximum understanding. So, metaphors are basically conceptual aids to understand abstract entities in terms of concrete objects, thus helping people make sense of the complex, dynamic surroundings. Much of this is predicated upon the “embodied mind” notion of human bodies (and almost symbiotically connected mind) having physical experiences in a spatial orientation, which affects the perception of reality accordingly from that viewpoint. Two digital examples enforce this.

The most popular computer metaphor is the desktop GUI (graphical user interface) model of interaction, with files, folders, windows, and even a trashcan; this model makes a visual computing environment more comprehensible and accessible to office workers by relating it to real-world references in the office space, to some degree of fidelity.

Another metaphor commonly used for enterprise software is the “dashboard” interface for showing multiple status indicators and performance metrics constantly updated in real-time within compartmentalized visual regions, much like an automobile's dashboard: speedometer, odometer, fuel gauge, etc. This enables better sense-making of complex data, by making it seem like a more familiar form.

Thus, a metaphor operates through a mapping of conceptual domains, to facilitate the interpretation of the unfamiliar in terms of the familiar. In so doing there is attribution of values, behaviors, and styles from the concrete to abstract, (and vice versa) to ensure appropriateness of fit by the participant. Again, the example of the desktop GUI comes to light as a more human recognition of what is essentially 1's and 0's, or basic machine code. In some cases the metaphor “breaks” due to faulty mapping. Indeed, this is often the case for mismatched affordances of a design: mapping cues don't fit the user's expected sense of how a design should behave. Referring back to the desktop interface, the notion of “minimizing windows” or “folder directories” somewhat breaks from the actual real-world references of file cabinets and tabletops.

Finally, the notion of frame figures into Lakoff's theory of metaphors in that frames set the overall perspective to help shape the meaning of a given linguistic concept. This is commonly seen in political rhetoric, with the “spinning” of controversial issues towards a specific advantage, by framing the debate using certain wording favorable to the intended listener. For example, politicians may frame the hot issue of taxation by using the words “relief” or “breaks” to suggest taxation is an adversity or pain to be relieved from.

In the world of Interaction Design, the use of mice and keyboards frames the model of interaction with visual computing environments. This hardware sets the assumption of drag-and-drop, point-and-click, type to enter values, and right-click to trigger a context menu, aptly drawing a certain kind of discourse with the GUI. Another example, the new Xbox360 gaming system, uses the phrase “Ring of Light” instead of “Power Button” to romanticize the mere act of turning on the next-generation gaming unit. Microsoft's Windows XP uses a visually embellished “Start” button to convey that is the starting point for a journey of digital encounters, framing computer use as more than just a series of tedious computational tasks. And, for that matter, Apple's use of the “Happy Mac face” (half computer/half human) and uplifting “dial tone” upon starting up the very first Macintosh established a very human sense—far more inviting than a blinking text cursor!

The concepts of interaction and language are tantalizingly close in achieving a useful relationship for designers in their daily practice. For an interaction to make sense, it needs a language to communicate. For language to express meaning, that meaning is found in the interaction of parts with a cognitively alert, live being, along with a full sense of expectations and cultural or social values of the immediate situation. These concepts go hand-in-hand by necessity; one cannot exist without the other.

This reflexive relationship becomes more evident by looking at the affordances and constraints upon interaction of everyday real objects. Through affordances a design speaks to users, provoking or inviting an encounter to ensue. Its level of success depends on the clarity, appropriateness, and conceptual linkage of the affordance to user's goals and expectations.

Affordance has gained popularity in HCI circles, primarily through Donald Norman's thoughtful analysis of the usability of everyday objects (chairs, doors, thermostats, tea kettles). Affordance refers to the “property of an environment that supports action to appropriately equipped organisms.”⁷⁹ For example, a chair's typical L-shaped structure affords people sitting (since an able-bodied person can naturally bend legs and torso easily), while a door's knob affords turning to open (since I have the requisite functional hands to turn the knob—for a person whose hands are full of grocery bags, however, the knobs are a hindrance). Going further, affordance, according to HCI scholar Paul Dourish, is a triadic relation among environment, organism, and activity focused on the notion of “being in the world,” acting in embodied space to accomplish some goal. ⁸⁰ Affordances are a form of communication, telling the user what is possible with a design—and constraining him to that possibility by virtue of materials, mechanics, and so forth.

Consider the shift in PDA (Personal digital assistant) devices from Apple's pioneering Newton to recent models from HP and Palm. These devices feature a stylus for pen-based interaction, resembling the natural feel of writing and gesturing with a pen. The device and screen afford tapping, writing, drawing, all the expected motions of pen-based manipulation, aligning with the intended metaphor of a portable digital notepad that frames the interactive encounter. However, the PDA software constrains the writing to a specific symbol lexicon (Graffiti), to approximate handwriting recognition.

A web-based business application (like Oracle Financials or SAP Supply Chain) conveys its affordances via consistent visual interactive elements: icons, buttons, links, tabs, menus, and various states thereof (hover, depressed, unselected, disabled, etc.). However, it's often not quite clear initially how various affordances relate to content and functionality, or the user's tasks—hence the challenge of virtual information spaces, with potentially N-layers deep of data and navigation. Site maps and directories provide some guidance. But through prior familiarity with standard web widgets, users armed with a mouse know that a dropdown list affords click and scroll to select a choice. Similarly, tabs have come to mean compartmentalized regions of content/functionality to be clicked to access that area.

BMW's iDrive is arguably the most ambitious attempt at mapping literally hundreds of possible actions and modes to a single control whose one affordance is turning and clicking a dial—while operating a two-ton vehicle at high speeds or in congested traffic. The invisibility of relationship between form and function (and the user's attempt to decipher it while driving) contributes to the high frustration and poor usability.

Such examples suggest how meaning arises in the use of the product, by virtue of understanding the affordances, constraints, and mappings of form and functionality. User interaction occurs in a particular manner as constrained by the materials and mechanics of the form and features, which should be visibly mapped to user's model of expected behavior, with real-time apprehension of consequence and feedback. Thus the user can modify her actions accordingly to achieve the optimal expected result: listening to music, jotting down notes, submitting financial reports, or adjusting the temperature.

The intersection of the core concepts—interaction and language—suggests the idea of design as an overarching platform for communication, the exchange of information from sender to receiver given a certain context. Indeed, the act of designing a product is a socially communicative act, of delivering value to an intended target audience, so as to evoke some response—emotional, physical, psychological, even social. On various levels communication can be regarded as a method of argument, grammar of symbols, or projection of sense/value upon the world.

Design scholars Richard Buchanan and Victor Margolin have proposed a rhetorical approach to design, as a humanistic activity dedicated to productive inquiry and making. Thus, it is an architectonic art, a systematic integrative discipline of forethought, a master art of strategic thinking that organizes and structures patterns of thought into new and powerful ways; narrative and argument are core aspects. Rhetoric is concerned with discovery, invention, argument, and planning. At its heart is the issue of persuasive communication, in which each designed entity embodies a well- formed argument. This view of the product as an argument suggests that the designer is like a speaker composing a speech, to be delivered to a specific audience. A product design is a situated act, relying upon balanced, nuanced relations known informally as the “Rhetorical Stance,” from rhetorician Wayne Booth. ⁸¹ Only through interdependence of the Classical elements (logos, ethos, pathos) can a speech be effective. Balance is found when it can change minds. An overemphasis on logos, ethos, or pathos will result in awkward, unnatural performance that does not achieve its goal of audience persuasion (i.e., move the user to favorable action). This argument is comprised of specific parts (drawn from Classical concepts initially outlined by Aristotle, Cicero, and other rhetorical thinkers), that should be held in balance: the logical structure of rational components (logos), human affordances and ergonomic qualities to ensure value (pathos), and a tone of voice or style (ethos). Rounding out is another aspect of narrative, or mythos, that conveys a unifying sense of story or plot structure for how the object fits the user's scenario. ⁸²

Briefly discussed earlier, semiotics was first conceived as a science of signs and symbols for communicative purpose. Semiotics explains the principles that underlie the structure of signs and their utility within messages. Valuing semiotics helps designers decide how to select and arrange the elements that comprise the message (buttons, links, icons, animations, etc.). Recent scholars suggest that a sign is the result of a signing process (known as semiosis), involving a balanced relationship among three critical elements: the signifier (which is some representation of an object, like an icon or symbol or image), the signified (the object which is being represented visually), and the interpretant (the human being interpreting the relationship of signifier and signified, imbued with his own set of social, cultural, and experiential background in the deciphering).

Mullet and Sano, authors of Designing Visual Interfaces, explore this in-depth with tangible examples, geared towards communications for screen-based computing environments. By clearly discussing atomic, essential principles, (like clarity, restraint, hierarchy, balance), they provide basic heuristics on effective visual communication techniques that designers can apply to develop their own sense of visual language for a given situation. Taking this approach, then, suggests that meaning is heavily dependent on the notion of interpretation, as a process of “reconstructing the meaning of a sign by identifying the sign object and grasping the significance of the connection.”⁸³ From this view, meaning comes from a person's interpretation of signs.

Complex financial business applications (accounting, cash management, taxation) have several hundred visual icons of varying subtleties of distinction, given the different mode or functional area of the application. Interpretation will vary per user type, task setting, goal, and expectations of the user. Some icons are clickable, others are status only indicators. In interaction and usage, the user will learn the differences.

The Xbox 360 game controller has a pre-set layout of buttons, marked with letters and in primary colors (red, blue, green, yellow) whose meanings vary per game—sometimes within the different stages of a game itself, depending on player mode (in EA Sports' NFL Madden 2006 Football, the modes change depending on whether a player is passing, kicking, running, or tackling). There is a mapping of the symbol on the gamepad to the virtual player capabilities, shown on the game screen.

HCI scholar/scientist Paul Dourish has hypothesized a different take on Interaction Design, that he terms as “embodied interaction,” a new model of interpreting interaction that extends recent HCI research trends in “tangible” and “social” computing. ⁸⁴ Dourish's argument is based upon the philosophical framework of phenomenology which is the study of experience and existence, that are intuitively felt and known by factual presence in the world. Dourish contends that embodiment is more than a physical property but is about social presence and participatory status in the world, having an (inter) active role in changing and becoming. Everyday engagement in daily activities and task completion is another core tenet; the setting of action defines the value and manner of the action. Thus meaning emerges from the participation of an individual agent with some object within a setting—a constant negotiation or conversation unfolding. It is formed continuously and interactively, in real-time action/location; meaning is not simply projected or found but instead created and shared through engagement with the artificial. ⁸⁵ This is a profound view of interaction that shifts the emphasis from the designer crafting the argument, or the interpretation of images, towards the place of action between the user and the object in question, given a situation and the particular lifestyle of the user.

This view encourages the designer to regard design as a participatory activity, not simply dictating to the user, but allowing the user to evolve and shape the encounter so it is a co-creative opportunity. Indeed, this view presupposes that the user can manipulate or improvise the design to suit her needs at the moment, as recently suggested by IDEO designer Janet Fulton Suri, in her account of everyday actions, Thoughtless Acts. Suri's work explores what occurs when ordinary objects are re-cast for impromptu purposes—for example, using your suitcase as a seat at an airport internet kiosk.

Another example to consider: Videogame interaction is a highly complex form of communication and engagement, whose meaning arises from the immediate, real-time encounter between the player, the controller, the game console, and the video imagery on the TV display. There is a coalescence of game play, game mechanics, and game interface that constitute the total value of the interaction, its meaning in terms of responsiveness of game interactivity and how it fits within situation/context of leisurely activity. There is learning, pleasure, frustration, and overall struggle and resolution in that continuous, unfolding moment of participation.

Thus, in summary, through the intersection of interaction and language, design becomes a platform for communication. Viable, actionable communication can occur from a variety of viewpoints: rhetorical, semiotic, or phenomenal. There are certainly others but these specific views sufficiently capture key issues of influence, interpretation, and engagement that characterize an interaction. In guiding the designer who seeks an effective communication-oriented solution, these views parcel out finer issues for debate and iteration. These are simply ways to perceive how meaning comes to be in interaction, when regarded as a communicative activity. In actual practice, however, an interactive encounter (and thus meaning itself) combines all three views into a dynamic, self-sufficient, whole user experience.

We have taken a path through the nexus of interaction and language to understand how to create products that deliver positive value to users, and thereby implicitly suggest a broader cultural backdrop of experience. Interaction shapes the perception of reality. A coherent and consistent system of interactions within the framework of design suggests a language of relationship building between people (user + designer, user + other users) mediated by the designed artifice. Value and meaning are deliberated, interpreted, and created via the interactive encounter, at multiple levels: emotional, cognitive, physical, visual. This activity (construed as a conversation or dialogue) characterizes the user experience of an artifact, which can proliferate and aggregate to impact society and culture at large, shaping values, norms, beliefs, attitudes, expectations, or standards of what is acceptable or appropriate. One's way of life or lifestyle itself can be influenced by well-informed Interaction Designs, to yield a satisfying, memorable quality of experience—one that can be shared, repeated, and enhanced.

Copyright for this article is held by Uday Gajendar; reprinted here with permission.