Contribution
Evaluation of Audiovisual Scene Analysis in Virtual Reality Classroom Scenarios
* Presenting author
Abstract:
This study evaluates audiovisual scene analysis for multiple simultaneous speakers. It enhances the realism of an existing test paradigm referred to as "audio-visual scene analysis" by Ahrens et al., originally developed for Virtual Reality (VR) hearing research.To achieve this, a modified version of the test paradigm from a previous study is employed to vary acoustic and visual scene representation factors and investigate their impact on audiovisual scene analysis. The revised paradigm simulates a classroom-like scene with increased realism in terms of auditory and visual aspects, where subjects listen to up to ten stories narrated simultaneously by 20 different speakers arranged around the listener in a circle of chairs. Participants had to identify the content of the stories and assign these to the respective talkers.Cognitive performance measures, including task performance, time needed, and NASA RTLX ratings are obtained after every of the nine trials per experiment. Two visualization types were used: 360° video and Computer Generated Imagery (CGI), and two acoustic representations, diotic and dynamically live-rendered binaural synthesis. This contribution presents the results from three subjective tests, with 360° video combined with diotic and binaural, and CGI combined with binaural audio.