Multimodal Communication in Virtual and Face-to-Face Settings: Gesture Production and Speech Disfluency

Authors : Burcu Arslan, Can Avcı, Demet Özer

Pages : 349-363

Doi:10.26650/SP2024-1443983

View : 23 | Download : 15

Publication Date : 2024-12-16

Article Type : Research Paper

Abstract :The COVID-19 pandemic has made online data collection a popular choice. It is important to evaluate how comparable online studies are to face-to-face studies, particularly in multimodal language research where modes of communication significantly impact the results. In this study, we examined individuals’ rates and patterns of speech disfluency and gesture use across face-to-face and online videoconferencing settings as they described their daily routines (N = 64). We asked whether and how multimodal language is affected across different communication settings and gesture use, particularly iconic gestures, is associated with speech fluency regardless of the context. Our results have showed that the participants’ overall disfluency rate was higher for the speech communicated via videoconferencing than the speech communicated face-to-face. However, the type of disfluencies changed across contexts, such that filled pauses and repairs were more common in online communication, whereas silent pauses were more common in face-to-face communication. These f indings signal an interplay between the cognitive functions of different disfluency types and communicative strategies. Results indicate that the overall gesture frequency and iconic gesture use were similar in both settings. Furthermore, the use of iconic gestures was found to negatively predict the overall disfluency rate, regardless of the setting. This finding suggests that using iconic gestures might facilitate cognitive processes, paving the way for a more fluent speech. This study demonstrates that multimodal language and communication strategies may vary across different communication settings and nuanced understanding of the differences in multimodal language between online and face-to-face communication can be gained using different contexts. The findings contribute to understanding the impact of increasingly widespread online communication on multimodal language production processes and provide foundation for future research.
Keywords : Jest kullanımı, konuşmanın akıcılığının bozulması, çevrimiçi iletişim, yüz yüze iletişim

ORIGINAL ARTICLE URL