First writer: Megan Grandmont
Second writer: Danah Hashem
Revision Feedback by: Tanya K. Rodrigue, Anne Mooney, and Kate Artz
Edited by: Julia Bennett
In “Sound Matters: Notes Toward the Analysis and Design of Sound in Multimodal Webtexts,” Heidi McKee (2006) identified what were then generally understood to be the three main elements of sound in multimodal texts: “vocal delivery, music, and special effects,” to which she also proposed adding a fourth element, silence (p. 337). In our discussion of sonic strategies on this webtext, we have likewise identified these four primary sonic strategies, which we have called voice, silence, sound effects, and music. At the end of “Sound Matters,” McKee pointed toward a need to further investigate the relationships between and among these four strategies, as well as their relationships to other modes they may be deployed with. In the intervening years, there has been an increasing scholarly interest in sound as a heretofore underutilized semiotic resource in multimodal composition. Much of this scholarship has worked to either situate sound among other modes (primarily image and text), or to further explore the affordances of the individual strategies named above (silence, music, voice, and sound effects) (Anderson, 2014; McKee, 2006; Shipka, 2006; Stedman, 2013; van Leeuwen, 1999). Approaching from an alternative perspective, a particularly insightful and compelling body of research has developed from a diverse interdisciplinary base that investigates interaction between and among these sonic strategies as modal affordances unique to sound.
As the disciplines that draw on the affordances of interacting sound strategies vary widely, they constitute an articulate and engaging “interdisciplinary body of perspectives” on how sounds dialogue and interface with one another to produce certain results (Anderson, 2014). In performance studies, Bruce Johnson (2005) examined Shakespeare’s use of sound interaction in Hamlet, claiming that Shakespeare intentionally employed music and song alongside the voices and sound effects to create “an acoustic experience” intended to function “as the major bearer of meanings” in the work, beyond the meaning of the text (pp. 257, 259). Sound artist Bill Fontana (2008) studied the interaction of soundscapes by creating sound sculptures, or arrangements of sounds for aesthetic purposes, which were designed to interact differently with the ambient sounds of the various environments in which they were installed, generating “new acoustic meanings” in each location (p. 154). Musician Dominic Robson (2002) explored the digital incorporation of nontraditional sound effects into music “to create very rich musical or sonic interactions” and to “make the sound output richer by building more layers of sound” (p. 60). Drawing on linguistics and musicology, Theo van Leeuwen (1999), in Speech, Music, Sound, discussed speech and music as existing along a continuum from sequential (involving turn-taking) to simultaneous. The interactions that take place both in turn-taking speech and musical antiphony create “a sense of difference or opposition between...individual voices or groups of voices” (van Leeuwen, 1999, p. 71). These scholars, along with many others, have essentially engaged with sound interaction as an important fifth sonic strategy that examines the relations and intersections among the other four. Cynthia L. Selfe (2014) has also hinted at this fifth sonic strategy when she recently defined aurality as “a complexly related web of communicative practices that are received or perceived by the ear, including speech, sound, and music” (p. 138, emphasis added).
With myriad ways of understanding sound interaction, our composers exercised the recursive skills of playfulness, flexibility, and reflectiveness to determine how to best engineer interacting sounds in their compositions.
Being Playful
As part of our respective composition processes, we explored different ways of using sound interaction as a fifth sonic strategy in order to accomplish our specific rhetorical goals. Robson (2002) described layering as a way to “explore the nature of sound and music together” (p. 60). Similarly, in his chapter “Composing Multimodality,” Joddy Murray (2013) identified layering and juxtaposition as two of the major values to consider when crafting a multimodal composition. His identification of layering supports Robson’s use of interacting sounds in a single composition, while juxtaposition creates a pointed “dialogue” between two types of compositional components (Murray, 2013, p. 339).
TranscriptM.P. Carver and Amy Zimmerman both applied Robson and Murray’s idea of layering to their audio compositions. In her piece “Laika,” M.P. layered guitar music with her sound clips and textual narration, experimenting with different volume ratios and styles of music. In so doing, she unexpectedly discovered that adding a second layer of guitar smoothed out the previously uneasy relationships among all of the layers.
TranscriptIn “Video Games: Not Just for Kids,” Amy layered sound effects and music with the narration. These layers, such as the booming reverb effect over the female voice and the Super Mario Bros. theme song underscoring the narration of the male voice, were intended to juxtapose the belittling “Mom” voice with that of the male narrator. This combination of voice, sound effects, and music constituted a complex emotional appeal that rhetorically aligned listeners with the male narrator and alienated them from the female character.
While still employing Robson’s (2002) idea of layering sounds and music, Danah Hashem also investigated Johnson’s (2005) concept of using voices, effects, silences, and music to create an “acoustic phenomenon” that would cohesively communicate meaning (p. 259).
Transcript In her audio journal “A Week in March,” Danah layered a variety of recordings from her daily life in an arrangement that would suggest a particular theme important to the recreation of that specific memory. For example, in recreating the hurried but well-loved morning routine of waking up with her husband and dogs, she layered narration over various sound effects, conversations, and non-verbal exclamations to complete the “acoustic experience” of the morning being recounted (Johnson, 2005, p. 257).
Being Flexible
While the composers worked to knit their various sonic strategies together into a single, fluid piece, some ran up against constraints and were forced to consider creative alternatives to their original plans. In weighing potential alternatives, many of us relied on concepts applied more generally to multimodal composition, such as Murray’s (2013) notions of layering and juxtaposition. These themes of layering and juxtaposing sonic strategies featured heavily in our responses to some of the rhetorical challenges that emerged as we composed.
TranscriptMegan Grandmont relied on both layering and juxtaposition in her audio drama “Night Visions” as she struggled to integrate the sounds of a dreamscape into the daily lives of her characters. She wanted to clearly convey the dreams in her drama as different from the rest of the dialogue so that the audience could recognize them as something separate from reality; she found, though, that the dream sound effects could easily sound cartoonish or simplistic when heard alongside the naturalistic sounds. In order to eliminate this overly simplistic impression, Megan implemented careful audio layering, slowly increasing and decreasing the volume of each dream sound over the dialogue and building this effect over time. By using layers to establish the desired juxtaposition between her two thematic soundscapes, Megan reshaped the artificial impression of the dream sounds, adding “complexity and nuance” to the plot of her drama (Murray, 2013, p. 339).
While employing layering and juxtaposition, Megan found it necessary to consider the notion of density as described by Brandon Jones (2011). Jones proposed density as a governing principle of web design; however, his discussion of density is extremely productive when applied to interacting sounds in an audio composition. He described density as a kind of “Goldilocks” proportioning of visual elements; too many elements can make a text feel “‘heavy’ and cluttered,” while too few elements on a page can “lose...relationships to one another.” The quest for density is therefore one that tries to find a balance among textual elements that is “‘just right.’” As Megan deployed audio layering to juxtapose her two soundscapes, she remained conscious of the sound densities she created so as not to build a composition that would be too dense for the listener to comprehend.
Jones’ (2011) concept of density proved relevant for Anne Mooney, as well, as she executed van Leewen’s (1999) audible sense of opposition between the two characters in “Claimed Experience: Owning the Past.” In this non-fiction audio drama, Anne read the lines for both her younger self and her present–day self. While it was rhetorically powerful to have a single performer playing both parts, using only one actor’s voice also made it difficult for the listeners to discern a sense of difference and interaction between the two characters.
Transcript To clarify when a shift in speaker occurred, Anne joined Amy, M.P., Danah, and Megan in placing layers of sound, in this case sound effects, such as turning pages and the scratching of a pencil on paper, over the dialogue. In doing so, though, she carefully considered sound density in order to find the appropriate balance in volume and overlapping sounds that would render the sound effects more than just background noise but prevent them from overpowering the voice. Anne elaborated on her original plan for the piece by increasing the sound density at certain points through laying sound effects over the dialogue, illuminating the characters’ opposition to one another in a meaningful way.
Being Reflective
After completing our projects, composers reviewed their work, shared it with the wider group, and assessed the rhetorical effectiveness of their interacting sonic strategies in the creation of meaning.
In reflecting on her choices with regard to sound interaction, Kate Artz faced challenges similar to Anne’s in that the characters in her audio drama “The Conversation” had to engage with one another in authentic ways; however, in a unique difference from Anne’s work, only one of the voices in Kate’s project could be heard. Kate’s audio drama captures a phone call in which two characters are conversing, but only one character’s voice is audible. The lone voice we hear, performed by Kate, interacts with a silence that indicates the presence of a second voice on the other end of the phone. The audience found that Kate had designed the turn-taking between the silences and spoken lines in a way which believably engendered van Leeuwen’s (1999) “sense of difference or opposition between...individual voices” without ever relying on an actual second voice (p. 71).
Sound interaction functioned much differently in Megan’s audio drama, resulting in a piece much like one of Fontana’s (2008) sound sculptures. Just as Fontana positioned one soundscape within another to create beauty and meaning in the divergence between the two, Megan constructed a dream soundscape to intrude on her representation of daily life; the dream world slipped in and out of listeners’ consciousness while the plot unfolded. The audience found this interaction between the two soundscapes both effective and compelling.
Conclusion
The rhetorical strategies of voice, silence, music, and sound effects have most often been investigated and explicated individually. In practice, however, our composers recognized a need to more comprehensively identify and describe the ways in which these rhetorical strategies were coalescing and operating together in our compositions. van Leeuwen’s (1999) discussion of “interacting sounds” considers the interactions of sounds within a particular sonic strategy, such as voices with other voices, but not the interactions of sounds between and among different sonic strategies (p. 66). We have therefore reframed van Leeuwen’s notion of “interacting sounds” as sound interaction as a means to analyze the interaction(s) between and among the four other sonic strategies. In this case, we have expanded upon a theory of sonic rhetoric based on needs derived directly from our practice.
Our understanding of sound interaction, which emerged out of necessity from our compositional practices, has become more robust through our recourse to other areas of scholarship besides sonic rhetoric, such as musicology and performance theory. Turning to these other fields presented us with the opportunity to apply concepts and terminology across disciplines in complementary and productive ways. It seems only appropriate that a theory of sound interaction should be further advanced through interacting with a wide variety of scholarship that explores sound in different ways.