Exploring Embodied Interaction beyond Text
Date:
Undeniably, all human interaction is multimodal in nature and both vocal and bodily-visual elements are central to meaning-making. In recent decades, most corpus-based studies of language and communication have focused on linguistic analyses of texts (Adolphs & Carter, 2007). Further, although conversation analytic studies since the 1980s has addressed the importance of visual modalities (e.g. eye-gaze, gesture, body positioning) in social interaction, we still know little about their sequentiality and position in talk-in-interaction, especially ‘how different modalities play together to perform recognisable social practice’ (Mortensen, 2012).
As part of the NUCASE (Newcastle University Corpus of Academic Spoken English) research group, this study draws on video/audio data of university students interactions in various settings and aims at a deeper understanding on academic interaction as embodied phenomenon. Adopting a dual approach, the study uncovers the interplay between embodied conduct and vocal speech from macro and micro perspectives. On one hand, a corpus-based multimodal discourse analysis enables us to investigate the under-explored phenomenon of gesturing in speech, especially the correlations between and the statistical patterns of discourse-marking functions of spoken language (e.g. discourse markers) and the co-expressed gestures. On the other, a fine-grained conversation analysis adds to the existing body of research on talk-and-bodies-in-interaction by focusing on the moment-by-moment shifts of participation framework as displayed vocally and bodily by participants in resolving the issues of who is to speak next and when to speak in multiparty conversations. The study therefore extends boundaries of studies on multimodal interaction with insights from the two alternative and complementary methodologies.
Adolphs, S. and Carter, R. (2007). Beyond the word: New challenges in analysing corpora of spoken English. European Journal of English Studies, 11(2), 133–146.
Mortensen, K. (2012). Conversation Analysis and Multimodality. In The Encyclopedia of Applied Linguistics.
Joint-presentation delivered by Hata, K. and Chen, Q.