The joint roles of movement and simulation in vocal communication of emotion

动作与模拟在声音情感交流中的共同作用

• 批准号: 341583-2012
• 负责人: Russo, Frank
• 金额: $ 3.42万
• 依托单位: Ryerson University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2015
• 资助国家: 加拿大
• 起止时间: 2015-01-01 至 2016-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=572310
• 关键词: joint; roles; movement; simulation; vocal

The proposed project is a continuation of our research explicating mechanisms involved with vocal communication of emotion, and embraces several methods that we have been developing over the past few years in the context of singing including movement tracking, gaze tracking, electromyography (EMG), and electroencephalography (EEG). Our research has thus far identified a small number of universal principles of song production that appear to stem from constraints of the human voice. Violation of these principles is effortful, leading to visible displays of emotional arousal such as exaggerated head movement. New experiments are required to understand the relation between different aspects of vocal effort, head movement, and facial animation. Head movement and facial animation of singers/speakers will be tracked in three-dimensional space (forward/backward, up/down, left/right) and with rotation about three perpendicular axes (pitch, yaw, roll). We have also found that observers track these movements and mimic them, suggesting that simulation may be an important aspect of perceiving vocal communication of emotion. However, our evidence of observer simulation has been limited to audio-visual presentations of song. New experiments will involve auditory-only, visual-only, and audio-visual presentations of speech and song. Observers' eye movements, head movements, facial muscles, and brain waves will be examined for evidence of simulation. We envisage that these experiments will reveal new insights about the joint role of movement and simulation in perception of vocal communication of emotion, with striking implications for understanding the dynamics of communication -- for example, how speakers/singers can use their voices to regulate engagement and the focus of visual attention in observers. Furthermore, benefits to Canada include the generation of basic knowledge that may impact the development of communication technologies (e.g., improved avatars for human-computer interaction), communication skills training, and therapies for special populations with emotion-processing deficits (e.g., autistic children).

拟议的项目是我们研究的延续，该研究阐明了声音情感交流的机制，并包含了我们过去几年在歌唱背景下开发的几种方法，包括运动跟踪、视线跟踪、肌电图 (EMG) 和脑电图（EEG）。迄今为止，我们的研究已经确定了一些歌曲制作的普遍原则，这些原则似乎源于人声的限制。违反这些原则是很严重的，会导致明显的情绪唤起，例如夸张的头部运动。需要新的实验来了解声音努力、头部运动和面部动画的不同方面之间的关系。歌手/演讲者的头部运动和面部动画将在三维空间（前/后、上/下、左/右）并围绕三个垂直轴（俯仰、偏航、滚动）旋转进行跟踪。我们还发现观察者会追踪这些动作并模仿它们，这表明模拟可能是感知声音情感交流的一个重要方面。然而，我们的观察者模拟证据仅限于歌曲的视听呈现。新的实验将涉及语音和歌曲的纯听觉、纯视觉和视听呈现。将检查观察者的眼球运动、头部运动、面部肌肉和脑电波，以寻找模拟的证据。我们设想这些实验将揭示关于运动和模拟在情感声音交流感知中的联合作用的新见解，对于理解交流的动态具有显着的意义——例如，演讲者/歌手如何使用他们的声音来调节参与度以及观察者视觉注意力的焦点。此外，对加拿大的好处包括产生可能影响通信技术发展的基础知识（例如，改进的人机交互化身）、通信技能培训以及针对有情感处理缺陷的特殊人群（例如自闭症儿童）的治疗）。