Many studies on various languages investigated the marking of information structure (such as focus-background and the degree of givenness of constituents) by prosodic means, in particular by changes in prosodic prominence (see Kügler & Calhoun 2020). There is also a large body of empirical evidence showing a close connection between gestures and information structure, e.g. in that manual gestures tend to mark focus domains and/or new referents (Ebert et al. 2011).
In fact, speech and gestures are often regarded as two parts of the same signal having similar but not necessarily identical functions (McNeill 1992). As a consequence, non-referential co-speech gestures are claimed to be prosodic in nature (Shattuck-Hufnagel & Ren 2018), with their beat component accounting for the rhythm and phrasing of movements of the hands, head or eyebrows. It is a matter of debate, however, which part of a gesture is the most relevant anchor of alignment with the speech signal (e.g. its stroke, apex or peak velocity), and how this alignment contributes to the production and perception of prominence.
Only a few studies have looked at the potential joint effects of spoken and gestural prosody on information structure, among them Türk (2020) on Turkish, Rohrer (2022) on English and Baills & Baumann (2023) on French. While temporal synchronization of the dimensions is confirmed for these typologically diverse languages, it is also shown that beat-like gestures are scarcely used on their own but only in co-occurrence with pitch accents when marking focus and/or givenness. This talk will attempt to give an overview of studies that investigated the division of labour between speech and gesture in the expression of multimodal prominence and their relation to different levels of information structure.
References: • Baills, F. & S. Baumann (2023). The multimodal marking of information status in French as a foreign language: What can we learn about the use of prosodic and gestural cues in an interlanguage? PaPE 2023, Nijmegen. • Ebert, C., S. Evert & K. Wilmes (2011). Focus marking via gestures. Proceedings of Sinn und Bedeutung 15, 193–208. • Kügler, F. & S. Calhoun (2020). Prosodic encoding of information structure: A typological perspective. In C. Gussenhoven & A. Chen (Eds.), The Oxford handbook of language prosody (pp. 454-467). Oxford Academic. • McNeill, D. (1992). Hand and mind: What gestures reveal about thought. University of Chicago press. • Rohrer, P. (2022). A temporal and pragmatic analysis of gesture-speech association: A corpus-based approach using the novel MultiModal MultiDimensional (M3D) labeling system [Doctoral dissertation]. • Shattuck-Hufnagel, S. & A. Ren (2018). The prosodic characteristics of non-referential co-speech gestures in a sample of academic-lecture-style speech. Frontiers in Psychology 9:1514. • Türk, O. (2020). Gesture, prosody, and information structure synchronization in Turkish [Doctoral dissertation].