Multimodality: Multimodal, Multisensory Communication
This section considers the differences students and instructors alike bring to multimodal communication and the ways that we can design for differences through accessible multimodal compositions. Just as Deaf composers redesign music videos to bring sound into the body, we can redesign our compositions and classroom spaces to access differences in communication, senses, and bodies.
As a Deaf composition scholar who communicates through multiple modes—sign language, speech, facial expressions, and visual gestures—I sense the different ways of using language in the classroom and the importance of accessing all the information that is being conveyed. My (too many) experiences watching uncaptioned videos and audiotapes—with sound recorded to such a low quality that an interpreter could not sufficiently interpret them, for instance—make me an instructor who designs my classroom with access in mind.
I consciously present information in multiple modes so that those in my class can access meaning in different ways using different senses. I show students ASL music videos, such as Sean Forbes’ (2013) “Sean Forbes ‘Watch These Hands’ OFFICIAL MUSIC VIDEO,” (see clip below) to help them recognize the value of communicating through multiple modes to increase the chances of reaching different audiences.
Any time we communicate, we use different—and often multiple—modes to express meaning to audiences. ASL videos show students that communication is more effective when done so in multiple ways to ensure that a message is being presented. They show that barriers between modes and between senses can be reassessed, and that differences provide more, not less, opportunities to communicate.
Showing composition students how Deaf composers synthesize sound, dynamic visual text, and performing bodies can help students recognize that they can communicate meaning through different and multiple modes. As Cheryl Ball stated in an interview, “highlighting multimodality… as a method through which we communicate—not only with writing, but with visual, aural, spatial methods too—helps give students a better way to interact, and engage, and communicate, and make change in the world” (Mahon, 2010, p. 114). ASL music videos show how modes can interact to create multidimensional experiences for a range of audiences.
To be effective meaning-makers in a multimodal world of differences, our students need to understand different ways of engaging with the “increasing multiplicity and integration of significant modes of meaning-making, where the textual is also related to the visual, the audio, the spatial…” (New London Group, 1996, p. 64). Analyzing how ASL performers communicate through visual, aural, and spatial modes can help students recognize how to use the elements of audio and visual design to communicate across differences and reach the senses.
ASL music videos can be incorporated into a multimodal pedagogy that “engages students in a multisensory experience” (Hocks, 2003, p. 644), and that teaches students “rhetorically based strategies for taking advantage of all available means of communicating” (Selfe, 2009, p. 644). ASL music videos embody multimodal and multisensory means of communicating through harmonizing visual text and bodies in movement. Through their aesthetic and rhetorical integration of multiple modes, ASL music videos support multimodal pedagogies that design for a range of bodies and differences.