Multimodality, Gesture, Culture
As a digital writing scholar and designer, non-dancer, and cultural outsider, I approach the videos from an angle of how they provide access to the dance narrative and aesthetic experience in different ways. Here, I consider the ways in which the videos intersect with conversations in digital rhetoric and writing: reading multimodal narrative; encoding embodied gesture; and accessing culturally specific performances. In the next few sections, I overview conversations on digital narrative, gesture, and culture that inform intersections between Kaustavi's motion capture videos and digital writing studies.
Multimodality: Reading Multimodal Narrative
Odissi has a long and complex history of narrative mediated across multiple media channels. As Kaustavi's section clarifies, Odissi dance, teachings, and narratives have been remediated from live dancing bodies to palm leaf painting, to written texts and stone sculptures, and reinterpreted from all these texts into the revived classicized form following a period of colonial suppression. As Guiseppe Getto, Ellen Cushman, and Shreelina Ghosh (2011) noted, it is important to approach any discussion of Odissi and digital media with an understanding of how this dance form has moved across media and the past, and the implications these changes have had for Odissi practitioners, to contextualize any discussion of remediating Odissi in digital spaces (see also Bolter & Grusin [1999] on remediation as a way of representing prior forms of narrative media). Furthermore, Odissi dance is deeply grounded in a narrative tradition, and scholarship on narrative across media (Ryan, 2004; Ryan & Thon, 2014) helps us consider the impact on this choreography's constructed narrative experience when the story's movements are told via entirely different sets of media resources.
In terms of the actual media artifacts themselves, Kaustavi's motion capture videos offer rich challenges for making sense of the ways they draw on multiple media channels to communicate an expressive message. Scholarship on multimodal semiotics long at the heart of digital media studies, particularly the work of Gunther Kress and Theo van Leeuwen (1996, 2001) and the New London Group (2000), provided language we can draw on for reading these motion capture videos as multimodal narrative. The New London Group (2000) divided multimodal communication into five main channels: visual, audio, gestural, spatial, and linguistic, with multimodal as the combination of these channels and a fusion of the above elements. Altogether, the motion capture videos draw significantly on all these channels in their design, although some channels are more salient than others (which, as we discuss in our analyses, significantly inflects the ways we interpret them as multimodal texts). Likewise, Kress and van Leeuwen's (1996, 2001) work on multimodal semiotics encouraged us to consider the range of decisions available to digital designers and to think about the affordances of one choice of media combinations versus another; with the three videos interpreting the same movement sequence, we can read these three versions together to interpret their varying ways of making meaning with greater nuance.
Gesture: Encoding Embodied Gesture
Dance scholars and writing scholars have very different stakes/approaches to thinking and writing about encoding expressive movement. Although the New London Group (2000) included gesture as one of its multimodal channels, for example, this channel has been less taken up in digital media composing classrooms than other channels such as audio, image, and visual communication. Kaustavi's discussion on encoding her expressive movements, as well as the historical context of the textualized Odissi body, digitally raises new ways of thinking about the implications of encoding embodied gesture for digital writing scholars. In connecting Kaustavi's motion capture work to conversations in digital writing studies, here we want to especially draw on two recent scholars' approaches to digital composing in ways that shed new light on encoding movement in a performance setting, specifically through the experimental captioning projects of Janine Butler (2018a, 2018b) and Sean Zdenek (2018).
Butler's work on integral captions is especially informative of our approach to thinking about negotiating access in considering dance across media. From a digital composing standpoint, Butler both theorized and performed different ways of translating embodied gestural communication into different channels, and building that into digital composing practice. In several recent pieces, Butler argued for integral captions, or captions in videos embedded into the video itself as a central part of the video's overall communicative work. In her (2018b) Rhetoric Review article, she framed her understanding of integral captions as "embodied and multimodal entities that make possible fluid connections across languages, cultures, and modes" (p. 287). Butler also called readers to think about what is marginalized and what centralized in developing captions (pp. 291–292), in ways that overlap with Priya Srinivasan's (2012) unruly elements of Kaustavi's project in putting the historically marginalized figure of the Mahari back at the center of her work. I point as well to Zdenek's (2018) recent work in experimental captions, in his recent Kairos webtext experiments with different ways of presenting captions that draw on different visual channels for communicating embodied data (in this case, vocal data, as well as background sounds). He, too, positions his project as one that pushes on boundaries of marginalized and central information: "Can (or should) we disrupt the hegemony of language in captioning? What non-linguistic or alternative resources are available to convey visual meaning for sonic events that are hard to caption, such as sustained sounds and keynotes?" ("Icons, Loops, and Overlays").
Kaustavi's videos are similar to Butler’s discussion of integral captions that are part of the video itself, or to Zdenek’s experiments with ways of incorporating different kinds of visual-textual captions into various videos. Both Butler and Zdenek experimented richly with different options for video composing and representing embodied information, pushing the boundaries of accessibility as currently understood in ways that build towards a deeper engagement with aesthetic performance and creative design. Thus, for me as collaborator and digital media scholar (as well as an outsider to both dance and Odissi traditions), Butler's and Zdenek's experimental caption performances provide frames for thinking about ways that expressive movement is communicated with attention to multiple semiotic channels via digital video design—attending to the ways that these designs foreground historically marginalized bodies.
Culture: Accessing Culturally Specific Performances
From a digital media writing standpoint, there are analytical tools available (as explored above) for making sense of these motion-capture videos as expressive multimodal artifacts and instances of experimentally encoded gesture. As a cultural outsider and non-dancer, however, there are many dimensions of the materials that I'm helping to share and design that I can't access. Odissi is a historically sacred dance form, performed in temples as an act of devotion by dedicated trained dancers, and Getto et al. (2011) especially emphasized the sacred nature of teacher and student in passing on the dance form. In our early conversations on Odissi, Kaustavi and I spent time discussing the classical Indian aesthetic concept of rasa, an aesthetic experience of an Odissi performance in which a performer communicates a particular emotional state to the audience, with religious, spiritual, philosophical, and theological overtones and implications. Such an experience, according to classical scholars, only comes from immersion in the tradition and training of the aesthetic sensibilities.
Although many Odissi performances today are public rather than religious events, certain sacred and historical dimensions remain inaccessible to the casual audience-goer. Whether or not these digital motion capture videos can continue to communicate these specialized aesthetic experiences to the appropriate audience is outside the scope of this project, but this question serves as a balance to the more formalistic analytical angles approached above—that Odissi is a dance form that goes far beyond any surface-level media analysis, and thus is best approached under the guidance of expert scholars, performers, and cultural insiders.
Altogether, these three frames of accessing the videos—via multimodality, gesture, and culture—inform my approach as digital media scholar in collaboratively working with Kaustavi to analyze and showcase these videos. From here, we move from our separate sections into dialogue on each of the videos themselves, finding common ground through the frames of "reading multimodal narrative," "encoding embodied gesture," and "accessing culturally specific performances," applying these frames to analyze the three videos. Although each frame provides a different analytical yield into understanding the video, we also recognize the ways in which questions of multimodality, embodiment, and culture significantly overlap. Thus, although each analytical section has a particular frame, we also draw on elements and scholars from the other two frames as appropriate to fully develop our discussions.