Capturing water movement

My works Reflets, Kabir Kouba and Vagues are composed with the inherent musicality found in the movements of water:

Reflets is constructed from footage of water reflections recorded at the Canal Lachine (Montréal, Canada) by a rainy, windy and cloudy day (see Figure 16.1) (Video 16.1).

Figure 16.1 Footage from Canal Lachine (Montréal, Canada).

The water is constantly moving and flowing, creating a homogeneous movement that becomes hypnotising. Our attention is drawn to textures: colour changes, caustic lines and little waves caused by wind. The surrounding sound is loud, mainly composed of heavy wind and the noise of the city.

The aural and visual recordings of Kabir Kouba present a great variety of water flux coming from the Kabir Kouba Cliff and Waterfall crossed by the Saint-Charles River (Wendake, Canada) (Video 16.2). With its intersections, diversions and downstream flow, this stream presents various aural and visual movements, from the calmest to the most powerful (see Figure 16.2).

Figure 16.2 Footage from Kabir Kouba (Wendake Reserve, Canada).

The materials used in Vagues contain movements recorded at Kabir Kouba (see Figure 16.2) and Lake Champlain (VT, USA – see Figure 16.3) (Video 16.3). This piece also includes visual directional movements, with clear beginnings and endings, like water splashes. Field recordings of water and environments relating to the sea, such as boats, birds, dinghies and the ambient noise of the harbour, are used. This larger variety of aural and visual movements used allowed me flexibility for the creation of musical gestures.

Figure 16.3 Footage from Lake Champlain (VT, USA).

Gesture and sound/image relation

I work with the movement of natural elements to compose relational progressions of sounds and images as they intersect, clash, distend or collide through time. By editing and combining aural and visual movements, it is possible, for example, to create musical gestures consisting of a river flow followed by a splash, a stream intersecting a sea wave, or two water trajectories clashing together. The sound of those gestures may come from water or not. The intention behind the sound/image relation is to give a viewpoint and to complexify the stimulus presented. In a major way, sound/image relations will lead the intention behind the musical progression. As they do so, they create expectations, directionality, suspension, emphasis or restraint. Musical gestures carry intentionality. Their meaning intuitively emerges in the mind of a viewer-listener via her/his imagination. This meaning is constructed by the way the viewer-listener’s body responds to the energy deployed by the organisation of musical gestures.

In Kabir Kouba, moving images are edited to establish a dialogue between each other. Movements of the water cross each other, then they separate to meet again. Extracted from the footage, but constructed by image editing and understated synthesised images (see Figure 16.4), the visual composition is entirely joined with the sound. The idea was to compose the music from the image, using the visual composition as a musical score. Along with field recordings of water, synthesised sounds are used to support, increase and imitate visual movements in the flowing water.

Figure 16.4 Stills from Kabir Kouba.

The sound acts as an accentuator for visual elements, bringing directionality to the continuous movement of the images. Conversely, the sound is often an echo to the visual movements. Sound can evolve by itself for a while and then come back in symbiosis with the image. It is a composition of meeting points and counterpoints between visual and aural elements that move together or independently and evolve through a floating and ephemeral time. Visual gestures affect the musical path with a transference of meaning going from image to sound. At the same time, aural gestures affect how visual material is perceived.

Gesture and causality

The organisation of musical gestures refers to actions, evolution and transformations. It concerns the application of energy and its consequences (Oliveira 2018). It is a combination of intervention, direction and progression over time; “a chain of activity links a cause to a source” (Smalley 1997: 111).

Gesture, in Smalley’s theory, represents a fundamental approach to structuring electroacoustic music. A musical gesture is related to the notion of causality, “where one event seems to cause the onset of a successor, or alter a concurrent event in some way” (Smalley 1997: 118). It relates to the communicative and expressive potential as an “articulation of continuum” (Wishart 1996: 17).

A simple example is to throw a stone into a lake. This action changes the state of the water, which was calm, by suddenly causing a series of waves around the entry point of the stone. This causal link (launching a stone that causes waves) is important for understanding how movement becomes gesture, and how the organisation of musical gestures relates to changes in energy.

As mentioned by Oliveira (2018), some examples of movement sharply contradict this idea of gesture. This happens when there is a limited or non-existent change in energy. In such a case, both sound and image lose their gestures to become textural.

Texture and the absence of gesture

Texture intrinsically comes from movement. But unlike gesture, it presents static energy, a rest, with a non-defined causality (Oliveira 2018).

My work Reflets is composed of two superposed visual movements: waves and reflections of a bridge in the water. The images are transformed to present the fine lines created by the reflections and the waves. Their evolution is caused by wind variations and clouds passing above (see Figure 16.5). The visuals are always changing but do not give a clear directionality. The wind behaviour is used as a dynamic model for the music, which is composed of synthesisers and piano strings manipulations.

Figure 16.5 Stills from the three scenes of Reflets.

The movement found in this work induces an experience of stasis, because it is always changing in a similar manner. The music generally follows the same path. The absence of gesture in the image is reflected in the absence of gesture in the sound, except for the three visual cuts that are accentuated by a sonic musical gesture. The sound/image relationship is used to accentuate and draw attention to textures, while supporting a contemplative musical form.

Oliveira (2018) points out that in both sound and image, energy is a consequence of movement. It is in constant transformation. The interaction between energy and what causes its release is similar to the gestural interaction between sound and image (Oliveira 2018). One gesture may inflect or trigger another one, terminate or bifurcate a flux of images and sound or, when successful, compose a discourse of sound and visual information that carry meaning, however diffuse or personal. Energy may be maintained, accumulated or converted, localised or diffused (Oliveira 2018). Sound and image moving together are energy through gestures and textures, shaping our perception of time and creating an alternate audiovisual experience.

Connotative and energetic meaning

I will discuss two kinds of meaning, as they relate to the natural environment: the connotative and the energetic. Similar types of meaning are discussed in Lehrer (2011) but my approach to the subject is slightly different – it goes beyond the purely musical aspect (in the conventional sense which mainly concerns rhythm and harmony) and does not directly address neuroscience studies.

The relationship between connotative and energetic meaning will help distinguish the codependence of narrative and abstraction, and will, in time compose a meta-narrative form of story-telling. In this meta-narrative form, the implicit meaning is provided by both the matter-centric imagination and the energy deployed by the matter. Connoted and energetic meaning are intertwined to form an arc carrying several emotions, representations and feelings. These meaningful connections are understood from the subjective perspective of the viewer-listener. The intention of the composer is not to control this. Implicitly or explicitly, the viewer-listener will make sense of these meanings.

The connotative meaning of a recognisable object includes the feelings and ideas that we may associate with that object. For example, a bird, a river or a tree usually refer to the real world of experiences. Our perception of the physical world is concerned by factors beyond the visual and aural recognition.

In Vagues, there is a short scene presenting a black frame and a seabird sound. A natural seabird transmits known sound, which is associated with the sea. A seabird is a real object within a range of physical shapes and colours. If a seabird is heard, most subjects will readily identify it and move on to explore other levels of meaning in the seabird. The subject will likely have personal experiences with seabirds, perhaps that of holidays spent by the sea or a passage on a ferry boat.

Moreover, the meaning depends on the relationship between all components of the work. It depends on the chain of sounds and images presented which will elevate the first impression of something else. In the context of Vagues, the seabird is obviously related to the water environment because various images of water are shown previously. According to this chain of relations, the bird may, for example, become a vector of freedom, alert or escape. Be that as it may, the cognitive relationship with the object has moved beyond physicality and stimulus. And the sense is thus established – it resides outside the object and inside experience, and of course, experience may vary from one individual to another.

Energetic meaning arises from abstraction and movement. It refers to colours, form, kinetic/morphologic characteristics, to the musical and its propulsive properties, and mostly, to the various relationships enacted by sounds and moving imagery. It emerges from the energy found in the aural and visual materials recorded, edited and manipulated.

Our brain is the screen

About moving imagery, the philosopher Gilles Deleuze affirms in an interview given in Les Cahiers du Cinéma (Bergala 1986) that “our brain is the screen – that is to say, ourselves”. I start from this statement to argue that our brain is the screen where the invented world – the work – unfolds. We don’t see its exterior envelope; we live it from the inside, immersed in it. This immersive experience unfolds when our brain accepts this self-referential organisation and signs a form of cognitive contract. We willfully abandon everyday reality to involve ourselves in fantasy, dreaming wide awake. On moving imagery, Thomas Zummer writes:

Any videomusic work proposes an invented place, the reality of which we, as viewer-listeners, contribute to defining. Through an informed acceptance of the artifices that make up that place, we construct meaning. The construction of meaning confronts our present experience of a work with the ensemble of our previous experiences of a similar aesthetic or cultural nature. The credibility of what is proposed depends on the coherence of the fictional universe we participated in defining. If only to ourselves, it must, ultimately, make sense.

But how does one create meaning with works mainly built from movements of natural matter? Likewise, where does meaning come from when there is an absence of explicit narrative? Intentionality, mental imaging, metaphors and symbolism are all part of the answer, and they are predominantly formed by each individual experience.

Intentionality

In his study Musical Intentionality: Between Objects and Meaning, Lee explores how intentionality structures the ways in which aspects of musical materiality relate to musical meaning. The author examines some of the basic structures of intentionality presented in Searle’s general theory (1983) and recalls that intentionality “denotes those aspects of conscious mental states that are directed at, are about, or represent states of affairs beyond themselves” (Lee 2016: 34). According to Lee, any “conscious mental state that has directedness – or ‘aboutness’ – to something beyond itself might be called an intentionalistic state” (Lee 2016: 34). Owing to the intrinsic subjectivity of mental representations, intentional contents and their objects are represented under unique aspects. On that point, Lee argues:

If we consider water as pure matter, using chemistry’s knowledge of molecules: it is H₂O (two atoms of hydrogen and one of oxygen). But water is not ordinarily experienced in this term. Rather, it is most commonly experienced as a substance with properties that far exceed its internal features. This way of approaching matter converges with Bachelard’s ideas concerning the imagination of matter. In Water and Dreams: An Essay on the Imagination of Matter, Bachelard writes:

With this poetic metaphor, the author points out how our experience with water is closely related to movement:

Located in movements and organised into musical gestures and textures, intentionality in my work is revealed by structural elements such as dynamics, rhythm, kinetic energy, colour, timbre, tone and grain. Aural and visual components are composed and composited to express, reflect or suggest feelings, thoughts, expectations and reactions.

The received meaning of such work is elicited from the organisation of gestures and textures and the way the materials relate to each other. The reception also involves the experience that a viewer-listener historically entertains with the matter presented. Often, the mental imagery directly links sound and moving image to tactile impressions, conceptual thoughts and all other kinds of sensory experiences. Humans need meaning and will construct it internally from the stimuli presented, in a mind-space associated with the mental image.

Mental imaging and audiovisuality

Following Bergson (see Deleuze 1966), an image is defined by Deleuze as the set of what appears (Deleuze 1986). In its cinematic manifestation, this set is a constant flux between intersecting parts in relation to a whole (Deleuze 1986).

The mental image takes on relationships, symbolic acts, intellectual feelings and is wrought with the thought of a new and direct relationship (Deleuze 1986: 203). According to Deleuze (1986), mental images are thought-figures that emerge from a chain of relations. As aural and visual movements of matter and objects unfold, the chain of relation corresponds to what appears to us. In other terms, images act and react on other images, providing meaning that is beyond the audiovisual content itself. Action, perception and affection are built in an interconnection of relationships. It is this chain of relationships that constitute the mental image (Deleuze 1986: 204).

The useful part of Deleuze’s idea is that perception is largely based on relations between individual objects. A sound associated with an image adds its own contribution to the idea of the mental image. In this context, the relationship between seeing and hearing is central to the process of mental imaging. Further, the relationship between all of our senses is connected in our mind. The relationship between seeing, hearing and feeling is thus at the heart of the process of mental imagery.

In her article On Stan Brakhage and Visual Music (2008), Maryline Brakhage explains the concept of “physiological relationship between seeing and hearing in the making of a work of art in film” evoked by Stan Brakhage. According to the filmmaker, the organisation of colours, lines, shapes and sounds “weaves a complexity of sensory truths to which we respond with our whole nervous system and deeply known physicality of being, as the sources of that knowing interact as felt response within the intricacies of mind” (Brakhage 1982, cited in Brakhage 2008).

A mental image is associated with a mind-space where our full body is engaged.

A mental image is a mind-space where we can hear, see, feel and act out our perceptions. A mind-space is where we can construct sense.

Metaphors and symbolism

To discuss meaning, we must pay attention to the way a viewer-listener talks about her/his experience with the work. Indeed, metaphorisation and reception processes are strongly linked.

According to Delalande (1998), who has studied metaphorisation in a purely sonic context, metaphors are constructed by a listener when s/he attempts to describe sounds. These metaphors happen at two levels. First, a listener can imagine descriptive metaphors to label different types of sound (related to the morphology of the sound), such as muted blow, splashing, impact, scraping (Delalande 1998: 39). When related to actions, these sounds act on each other, and, in a symbolic way, act on the listener. For example, wind in your hair, blows to the stomach or nails on a blackboard. Secondly, metaphors can be articulated into narratives or complex images in a more personal way (Delalande 1998: 27): like swimming in the waves, like walking in the sand. The metaphors used will likely differ from one person to another. The story of this non-story is the fuzzy affect that rises in the listener’s mind and body. As is the same with videomusic.

Ultimately, videomusic provides a musical experience enacted by both visual and aural materials. A visual gesture or texture reflected in the sound, or vice versa, is a metaphor. In Vagues, the high-pitched sound of the violin has nothing to do with the water splash seen. But the energy is mutually reflected, giving a meaning that goes beyond the simple relation between the movement of the water splash and the sound presented with it. The sound becomes the metaphorical meaning of the image. This meaning may be alarm, accident, fear, etc. There are many possible interpretations.

My practice focuses on the idea that the sound/image coupling becomes a symbolic link that articulates human experience with movement and matter. This approach, rooted in an observation of the natural environment, became particularly important in the composition of Vagues.

The meta-narrative arc is partly guided by the poem The sea, by Pablo Neruda. Here is an excerpt:

Inspired by the three last lines of the excerpt, the final scene of Vagues presents a symbolic element: a foot in the water (see Figure 16.6). In literature, water symbolises life, purity, growth, birth and rebirth, among other things. This final scene may reflect one of these symbols, influencing the way meaning is constructed for the whole work. I recorded this video with the intention to capture just the water waves. My foot appeared in the shot by accident. This occurrence defined my work on meaning and led to my personal way to address it.

Figure 16.6 Still from Vagues. My foot in the water.