When you think of a game, you certainly think of the visuals, which over the years have become more and more eye-popping. But how often do you think about how the sounds and the music of the game have affected your experience?
This chapter contains information on how music and sound can affect the game experience, including:
Games have had music almost from the beginning, and today the music we hear in games is often as compelling as the music we hear in movies. It is rich, complex, and relevant to the situation. In games, music even provides clues that help us know what is going on, themes associated with situations or characters, and great atmospherics.
In the beginning, however, the music of games had none of the depth and meaning that it does today.
The first game to make sound an integral part of the experience was Pong. The characteristically electronic sound of the “ball” hitting the “paddle” brings instant recognition to all those who played Pong and its many variants in the 1970s. Likewise, the music and sound effects of games such as Space Invaders, Pac-Man, and, later, series games, such as Mario and Legend of Zelda, provided music that is specifically associated with that game. All you need to do is listen to a few notes of any of these game themes, and you instantly remember the game and the experience you had with it.
This kind of imprinting and association was particularly effective when games used a repetitive eight-bar loop over the entire game, with maybe a few special sequences for entering a level, dying, and completing a level. But overall, you heard the same piece of music over and over again for however long you played the game.
Today, the situation is more complex. The eight-bar loops and simple “pong” sounds are gone, and the music and sounds used in games now fill different and more sophisticated roles.
There is a lot that can be learned from the movie industry and how it uses music. Film composers know a lot about how music can affect a scene, alter a viewer’s emotional reaction, convey information, or intensify a scene. But there is a difference between game music and movie music. In the movies, you sit in a dark room, a captive, passive audience, and you are immersed in the film’s world for two or maybe three hours. In a game, you may play for 10 hours or 50. In the case of persistent-world games, you may play hundreds or thousands of hours in that world.
Game producers are asking for variety, and scoring music for games can require hours’ worth of music—a far, far cry from the simple Pac-Man theme. But that’s only part of the challenge. In addition to the need for variety—new themes for new levels, each area having its own music, special music for special situations, and so on—there are the issues of musical tastes. Many players today prefer to turn off the music in the game, put in their own CD or set the music in some online player, and listen to what they like. In this case, the hours of composed music for a game go to waste.
There are ways to encourage people to listen to the game’s music and sounds. For instance, you can make the music and sound truly part of the interactive experience—meaning that the information they convey is not just window dressing, but is an integral part of the game experience. The player receives useful information that makes the gameplay experience better in some way.
For the rest of this section, we’ll look at some of the ways that music is used in games, and I’ll offer some information about how to approach each of these specific types of uses.
One way to look at almost any music is to imagine it as a combination of pace, intensity, and mood or style. These qualities are often intuitive to the composer, and it can be revealing to look at how these elements combine in different types of music.
For instance, in the following sections, I’ve listed a few of the possible types of music you might include in a game setting. Music that accompanies a peaceful moment would most likely be low pace, low intensity, and low in mood/style. In contrast, music designed to accompany battle or a chase or race scene would probably be high in all three qualities. Music that indicates tension or the threat of danger might be low pace, high intensity, and moderate in mood.
You can visualize this in terms of an X, Y, Z graph, where:
X = Pace
Y = Intensity
Z = Mood, such as military, pastoral, heroic, romantic—style of music
One of the key distinctions in game music is between music that is more or less decorative—meaning that it is just there to be enjoyed and to put some kind of background on the game—and music that is more functional, meaning that it connects more directly with the action or that it conveys some specific type of information to the player.
What I call background music is music that is generally unrelated to the action, but simply there to entertain and provide a musical accompaniment to the player. This music does not respond to changes in the action. Early eight-bar background loops were typical of the background music of games in the ’80s and early ’90s. Some games still use background music that has the general effect the designers want—whether it be heart-pounding action, eerie surrealism, or a pastoral and quiet mood (which is not too common in games).
A good example is an action, platform, or racing game that has a rhythmic, stirring bit of music for the background. It’s common to create a different theme for different levels or, perhaps in the case of a racing game, for different races, tracks, or routes. The difference between background music and situational music in this case is that the background music stays the same regardless of the player’s situation on that level or in that race.
Often, using simpler background music, a composer can still create a recognizable theme or melody that will become associated with the game in the player’s mind. The more a game is played, and the more often a player hears a particular theme, the more likely it is to imprint. However, it is also possible to use recognizable themes with situational music as well.
I contrast background music and situational music by one simple test: Does the music reflect what is actually happening in the game and does it change with changes in the gameplay? Movies almost always use situational music (though there are some exceptions). In movies, the music changes as the story unfolds. Sad moments are generally accompanied by sad music, heroic moments by heroic music, and so on. Anyone who has watched a few movies is already aware of these musical associations, whether consciously or not.
In this section, I’ll look at some types of music that can be related directly to the events of the game.
Music often relates directly to what the player is doing or to the situation around the player. Some of these are binary pairs—such as combat or non-combat, running or walking, and so on. Here are some of the places that the music can function seamlessly with the action of the game:
Peace (Non-Combat). The player’s character is not fighting or in the fighting mode.
Combat. The player’s character is fighting or in the fighting mode.
Danger. There is something dangerous nearby. Danger music acts like a sixth sense for the player—an early warning system, if you like.
No Danger. Apparently, there’s nothing to worry about. How reliable do you want this to be?
Fast-Moving Thrills. This music is used for chases, fast cars and vehicles, extreme sports, and the like—from snowboarding to bungee jumping to tearing down the middle of the road on a motorcycle, and so forth.
Running/Sprinting. You might play this music to enhance the sense of speed when the player is running. If there are different modes for running and sprinting, you might even have a change in the music when the sprint mode is activated.
Walking. This music might be distinguished from the running music to indicate a more sedate pace.
Specific Areas, Levels, Missions, Vehicles. This music is used for particular places or player POVs. For instance, there could be different music for each town you visit or each level you play. There could be special music just for caves or dungeons, and similarly, there could be music for plains, mountains, floating cities, and outer space. Vehicles, too, can have their own music, so that you might get something different in an old Buick than you would in a brand-new Ferrari and something still more different if you were driving a tank or flying a plane. In Grand Theft Auto, you not only have a sort of background music, but you can choose different radio stations when you drive a vehicle. This is brilliant because it allows the player to choose the kind of music he’d prefer to listen to—within the limits imposed by the designers of the game, of course. The possibilities are pretty much endless and depend mostly on the type of game and the type of scenes you are developing, in addition to your budget (of course) and the kind of effect you want the music to have.
Music for Spells and Magic. This music would be used for specific types of spells or magical conditions—especially, but not in any way limited to, music-based magic.
Can you make a scene bigger or smaller? Can you directly convey sadness, tragedy, wild elation, or outright terror through music and sound? You bet.
Besides its direct connection to functional game elements, music in games can emulate movie music and affect the emotional or psychological aspect of the player’s experience. See also Chapter 30, “Ways to Communicate with the Player.”
To begin with, it helps to consider the music the way a film composer would. The first question to ask is whose point of view does the music represent? Does it represent:
What’s on the screen?
The character’s point of view?
The audience’s (in this case, the player’s) point of view?
Let’s take a romantic scene and look at how this works. In one case, the characters are flirting, and there’s some real connection happening. Things are going well. This is number 2, the character’s point of view, and the music could reflect this by being romantic.
But then, imagine that the guy is a dork, the girl thinks he’s a loser, and he’s doing all kinds of things to impress her that backfire. Here, from the audience’s point of view, it’s probably pretty funny, so some comedic music might work. This would be number 3.
Now imagine that this courtship scene is taking place, but there’s an axe murderer hiding in the closet. The audience knows it, but the characters don’t. This is number 1, what’s on the screen, and the music would likely reflect the tension and impending danger of the scene.
These are some simple examples, but really, the choice of music can be more varied. The axe murderer could be lurking in the background, but the music stays romantic. What effect would that have? Or the music could stay comedic. Again, the effect is different. These sorts of choices are generally made by directors and/or composers to achieve the particular effect and convey a particular message to the audience, but it is clear that different combinations of events and music produce different audience (player) responses.
Again, borrowing from the movies, there are basically two main ways to score a film (though there is at least one less-common third way). The two main approaches are:
Character themes
Completely contextual
Star Wars is a perfect example of the first approach. Each main character—Luke, Darth, Leia, Han—has a theme. The themes are woven through the movie, varied in ways to fit the action, mingled with the ostinato (a sort of under-theme) in various ways to achieve all sorts of emotional connections to the action.
To take a closer look at this, suppose you have the hero and villain in an epic struggle—I think of Luke and Darth. You play the theme of the enemy with perhaps the under-theme of the hero, and you give the impression that the enemy is going to win. Shifting the intensity of these themes and mixing them in different ways will convey different information to the audience/player.
In an interactive sense, this thematic approach could actually be tied to the situation and to how successful the player is at any given moment in a battle.
Primary character themes fall into a few main categories:
Good guy (hero).
Bad guy (villain).
Ingénue (romantic interest, female lead).
Comedic.
Strong characters. (Characters who are very strong, but not necessarily the hero or villain, might get a theme—for instance, a dragon that is essentially a force of nature in itself, but is neither good nor bad.)
Sides (such as good guy/bad guy, but for whole group perspectives—Empire/Rebels, Fellowship/Mordor, and so on).
Using character themes in this way helps identify the action, especially when cutting back and forth, and creates a musical aura around the character. As one composer says, it’s a ballet of thematic material created as the editor cuts back and forth.
The second approach is a purely contextual one, in which the music follows the events of the story without regard to specific character themes. This is a very common way to score a film, and it can work very well in a game. In a happy moment, the music reflects the happiness. In a sad moment, it reflects the sadness, and so on. Of course, as I discussed before, there is a wide range of options possible within this approach, and the music does not necessarily have to be in a one-to-one connection with the events.
The first approach actually borrows from the second approach in that character themes can be modified, such that in a sad moment, for instance, Luke Skywalker’s theme music can seem sadder by changing the pace and the instrumentation, perhaps even altering the key from major to minor, and so on. In any case, character theme music can also be contextual.
A less common method of scoring a film is similar to the work of John Barry (who scored Out of Africa and Somewhere in Time, among others). This could be called the “Arc” approach, in which the score establishes an overall feeling that does not change with the action, but simply establishes a single mood and continues it, possibly with minor exceptions, throughout the film. This is also an option for games, particularly those that do not seem to require a more complex thematic or contextual approach.
Many games today have elaborate noninteractive scenes that must be scored as well. These scenes serve specific purposes to introduce, show important information, provide the player with special rewards, and establish certain dramatic flow from the beginning, key middle points, and the ending. Consequently, the music should fit the intent of the scene. In this way, it is likely to be very much like composing for movies, and the information in the previous sections should be useful.
For instance, a lot of games use big, heroic music to introduce the game. Some use dark, moody music. How much information can you convey simply with the way you handle the opening?
Surprisingly, the music composer is also responsible for an odd array of music that seems almost too inconsequential to be noticed, but it is just the kind of music that can help immerse a player in the world’s fantasy. Examples of this type of incidental music are:
Hold music when the player is on the phone.
Background TV playing something—could be soap operas, cartoons, a chase scene, or some schmaltzy drama.
Elevator music or the music in a department store or mall.
“Live” musicians on the street or in a pub or bar.
Car radios.
The object of this book is to encourage creativity and give you some resources from which to draw. The concept of how to orchestrate the music for a game is a very subjective topic, and one that will depend entirely on the tastes and intentions of the game’s designers, producers, and composers.
In other media, such as movies and TV, there are two meaningful ways to describe orchestrations:
Classical
Contemporary
The classical approach is familiar to all moviegoers—the large orchestra that seems bigger than life. It creates a sense of magic and can color a score with any emotion—heroic, romantic, funny, quirky, eerie, frightening, pastoral, exciting, mysterious, and so on. It can range dynamically from pianissimo to double forte (very soft to very loud) in the same piece. However effective and common it is, though, it does carry with it a traditional, dated quality.
In contrast, the contemporary sound—which can be dated more or less from the ’80s on—relies on urban music, techno, electronics, and contemporary styles. It is somewhat more difficult to establish the entire range and spectrum of emotion and imagery with modern instrumentations, but it isn’t impossible. Lacking a couple of hundred years of classical history and composition and lacking the first 70 to 80 years of film composing, contemporary scores must work harder to convey some of the more complex emotional associations. However, it does very well with many common themes, such as danger, heroism, action, and excitement. And it does convey an immediate sense of its era.
This strength of being recognizable from a specific era is also a potential drawback, because contemporary music can quickly become dated and even quaint or passé. Anything that’s recognizably contemporary has probably about a five-year window of freshness, after which it becomes dated.
Many composers approach the situation by using a hybrid of classical and contemporary, giving them access to the best of both worlds and the most versatility.
Some music is outside these categories, such as jazz and most ethnic/exotic music, including African, Middle Eastern, Indian, oompah, bar mitzvah/wedding music, big band or period music (such as Gregorian chant), and other music associated with a particular geography, ethnic group, or period in history.
Given that you have decided on the point of view to use in any given situation and the other parameters of your score, what are some other ways to affect the player’s experience? How much control do you have?
Part of the effect of music is cultural, so we recognize certain kinds of music as being sexy (soft jazz, saxophones, and so on) or romantic (swelling strings, perhaps), but these represent a cultural language developed, in some cases, over several hundred years. If the listener was born in the Congo or the Middle East, without exposure to the music of Western culture, he or she might have a different response.
Even within our cultural setting, people might have different responses to recognizable types of music. If you take an electric guitar with big vibrato and wah wah, you may instantly think of the ’60s and its associated music, but whether you respond with nostalgia or you think it’s funny and silly depends on your perspective. Another example is the accordion, which is a pretty cool instrument, but has been associated over the years with some decidedly staid and unhip music. For that reason, people generally hear accordions and have a negative reaction.
The “color” of music is established by a number of different factors, including melodic and harmonic choices, as well as instrumentation. It is pretty clichéd to use brass in 4th and 5th intervals for heroic music—the Roman trumpeters announcing Caesar, for instance. But in jazz, the brass is orchestrated in block voicing (where the harmony instruments always move in the same direction as the lead). And you would get a very different effect using flutes, woodwinds, or strings. Again, if you want it to, this can get very technical, but much of it is so ingrained in our minds that we know it when we hear it, and composers don’t generally sit around analyzing what they are doing. It is a language they have become fluent with.
Besides instrumentation, there are a lot of other buttons you can push, including how you use tempo, pace, meter, melody, and harmony. For instance, watch (and listen to) the Batman movie, The Dark Knight, and notice the driving force of the underlying music, which continually seems to build throughout the movie. The rhythmic theme is repeated again and again, with different melodies and effects over it. It moves to the background to create an underlying, increasing tension and to the foreground to create immediacy and drama. Like the Jaws theme, it’s simple and effective.
For trained musicians, there is a lot of information about the types of scales, modes, harmonies/chords, and melodic sequences that can reliably change a listener’s experience and mood. For instance, chords such as suspended and diminished chords tend to lend a sort of undefined quality to the music, while the dominant 7th begs for resolution.
Also, music can almost immediately tell us something about a character or theme. For instance, if you show someone walking down the street and you play in the background the steady “boom boom” rhythm of a rap song, the audience/player instantly has an impression of that character.
The job of the composer for movies, TV, or games is to push the listener’s buttons by first knowing what those buttons are, and then using them effectively.
Types of effective music include:
Romantic
Military/pageantry
Victory
Defeat
Creepy/scary/tension
Suspense/tension
Dark/moody
Big/huge
Small/intimate
Action (such as fast-moving sports or chase scenes)
Dramatic moment
Sad
Jubilant
Determined
Heroic
Indeterminate (what’s happening, anyway?)
Funky (being hip/with it/bad/fly, or whatever is current)
Pacey (responding to elements of timing in the story and events)
Comic/silly
Sounds and sound effects are used extensively in games, and while a lot of attention is paid to the latest graphics innovations, today’s games would be stale and somewhat empty without the many sound effects that enhance the player’s experience. In some of the same ways as movies and in some unique ways, sounds and sound effects are an integral part of a game design.
This section deals with some of the ways sounds are used in games, along with a few specific examples of each type of use.
Ambient sounds are used to set the scene and the environment. These are the sounds and noises that are characteristic to a location. For instance, a cave might have a certain low hum inside and echoes from various small events—bat wings, small stones falling, the splash of a fish in a subterranean river, and so on. In contrast, an office building might have a slight buzz from the fluorescent lamps and the whir of computers, the background ringing of telephones, and so on. A jungle might have the call of a peacock or a monkey’s chatter in the distance. Here are a few types of ambient sounds:
Crickets
Machine whir/hum
Distant roar of ocean or babble of brook
Gunfire in distance
Heavy machinery
Traffic sounds
Aircraft sounds (ambient)
Footfalls (different surfaces)
Room sounds (ambient, variable with acoustics of room or location)
Electrical discharge of force field or something like it
Voices (ambient)
Sounds are often used as clues for the player. In this, they are unique to interactive entertainment. The sound clues can be very artificial and obvious, such as a special beep issued by an item the player needs to find. However, they can be far more subtle—distant voices that get closer as you approach, warning you of possible danger ahead; the soft padding of an animal or the splash of a frog jumping into a pond, calling your attention to a half-hidden path. Designers generally don’t waste their time putting in animations or specific sounds without meaning. There are always ambient sounds and animations that are used frequently enough to become meaningless in terms of information, but those one-of-a-kind events almost always point to something useful or important. Here are some of the kinds of sounds that can be used as clues:
Voices
Beep or ticking of a clock—or is it a time bomb?
Rustle of leaves to indicate the presence of enemy or animal in forest
Sounds of breathing or other natural sounds to locate people or creatures
Sound of footsteps (player’s) to indicate some change in the ground or floor
Whistle or whine of an incoming shell
Gunfire to tell you you’re being shot at
Cocking of a gun as a warning
Breathing sounds to indicate whether the player’s character is getting winded/tired
Cracking sound as something breaks
Sound has become far more sophisticated as games have evolved, and now it’s entirely possible to give players a sense of realism and a better sense of location by creating sound that more or less realistically depicts the distance of the player’s character from the source of the sound. This can be used as a clue, or it can simply enhance the enjoyment of the game by making it seem more alive. The ultimate of this experience is to use surround sound technology to put the player in the middle of a real environment. Distance and surround effects can be used with any type of sound effect. In essence, the closer a sound, the louder it will be. The farther the player’s character gets from the sound source, the lower the volume. Of course, the player character’s presence alone might cause the sound to change, to become quieter, silent, or louder.
There are many unique, event-specific sound effects used in games. These are sounds associated with specific game events, such as the contact of a fist with someone’s face, a shovel digging in dirt, or the casting of a magic spell, and so on. They can also be any sound that is specifically associated with any event. These sounds bring the world to life and tie the actions of the player and other characters to sounds that provide more feedback to the player and a sense of continuity and realism.
Feedback is an important aspect of these event-driven sounds. It often provides an auditory method of checking that you have done something—for instance, a fizzled spell will sound different from a successful one. A missed punch will sound different from one that connects. In many circumstances, where a player’s attention is focused on many things at once, these sound distinctions can sometimes be the most important ways of knowing what has happened and what is happening.
A telephone rings.
Someone gets hit.
Any type of spell is cast. (The sound will vary.)
There is a knock on the door.
The door opens.
Someone walks across the room or along a path. Is it wood, stone, masonry, gravel, leaves, or something else?
The elevator rings the floors and dings when it reaches a selected floor.
Someone lights a match.
A rock is thrown into a pond.
Your character (or another) screams in pain or grunts.
Design Challenge
Spend some time listening to the music and sounds in your favorite movies. How do they use sound to influence you? How do the ambient sounds affect the scene? Make a list of what you discover.
Play some games you are already familiar with and do the previous exercise. Then answer these questions:
Does the game genre affect how the designers used music and sounds?
Which types of games used situational music?
Which games used sounds as clues, and how did they do so?
Think of five different scenes you might find in a game—preferably in your own games—and make a list of every sound that could occur in those scenes.
How would ambient sounds help or detract from your scenes?
Think of a game you want to make. What kinds of music would you want and why? Make a list.
Would you go with situational music? Why or why not?
Would you go with character themes? Why or why not?
What would be the effect of having no music at all?
Make a list of ways your favorite games used music.
Make a list of the ways your favorite games used sounds.
Can you think of an innovative way to use sound? Create a game concept that uses sound in a novel way—not just as a novelty, but as an integral part of a game you would like to play.