Contribution
Who’s turn is it? EEG Correlates of Visual Cue Interference in Speaker Prediction
* Presenting author
Abstract:
Conversations naturally involve speaker switches and visual cues, indicating turn changes, identifying the next speaker, and signifying when conversations should proceed. Predictive mechanisms may help individuals anticipate turn endings, with cues from facial expressions, head movements, and eye gaze often signalling an intent to interrupt or contribute. By clustering these cues based on their informational value, we gain insights to enhance realistic multi-talker animations in virtual environments. We hypothesize that mixing these cues, such as through manipulations of pre-recorded videos, will disrupt the ability to predict the next speaker. Our study presents EEG recordings to examine the neural processes involved in speaker switch prediction, comparing brain responses to observing congruent and incongruent visual cues for pre-recorded natural triadic conversations. Pilot data indicate that incongruent cues seem to produce a larger P300 amplitude relative to congruent cues. If this can be confirmed in the main study, this effect may reflect higher cognitive load or re-evaluation processes generated by the unexpected nature of the incongruent cues, as P300 is commonly associated with attention allocation and the updating of cognitive context. This finding highlights that visual congruence plays a role in supporting conversational flow by reducing the cognitive demands of predicting speaker changes.