Active Social Vision: How the Brain Processes Visual Information During Natural Social Perception

主动社交视觉：大脑如何在自然社交感知过程中处理视觉信息

• 批准号: 10608251
• 负责人: AVNIEL S GHUMAN
• 金额: $ 71.93万
• 依托单位: UNIVERSITY OF PITTSBURGH AT PITTSBURGH
• 依托单位国家: 美国
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-02-17 至 2027-12-31
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/10608251
• 关键词: Affective; Anxiety Disorders; Attention; Attitude; Behavior; Brain; Brain region; Code; Cognitive; Complex; Computer Analysis; Computer Models; Computer Vision Systems; Cues; Data; Diffusion; Emotions; Environment; Epilepsy; Event; Eye; Eye Movements; Face; Facial Expression; Family; Frequencies; Friends; Future; Goals; Head Movements; Hospitals; Hour; Human; Image; Implanted Electrodes; Laboratories; Learning; Machine Learning; Maps; Measurement; Measures; Mental disorders; Methods; Modeling; Monitor; Mood Disorders; Motion; Motor; Neurobiology; Neurocognitive; Nurses; Operative Surgical Procedures; Parkinson Disease; Participant; Patients; Pattern; Perception; Persons; Phase; Play; Population; Post-Traumatic Stress Disorders; Process; Property; Recording of previous events; Relaxation; Role; Saccades; Sampling; Schizophrenia; Signal Transduction; Social Behavior; Social Environment; Social Interaction; Social Perception; Stimulus; Structure; System; Techniques; Testing; Time; Traumatic Brain Injury; Update; Variant; Vision; Visual; Visual System; Work; active vision; autism spectrum disorder; brain surgery; computer framework; experience; experimental study; gaze; information gathering; information processing; model design; movie; multisensory; nervous system disorder; neural; neural model; novel; operation; prototype; response; sample fixation; scaffold; social; social deficits; social neuroscience; spatiotemporal; visual information; visual neuroscience; visual process; visual tracking

Summary/Abstract The dominant paradigm in social visual neuroscience has been to present simplified stimuli, often consisting of static images in isolation presented without context, passively, to subjects required to maintain gaze fixation. While much has been learned using this paradigm, it cannot capture all aspects of how social vision (SV) works. Specifically, active observers combine motoric information gathering behaviors (eye and head movement) to optimally sample perceptual information in a complex, natural multisensory social environment. Indeed, there are few places where active vision is more critical than in the study of the neurobiology of SV. Real world SV is a prototypic active sensing process where we use our experience, along with all of our sensing and interpretive capacities to gather social and affective information from other people in a complex and dynamic setting. Simply put, SV, as typically studied, does not adequately approximate actively interacting with friends, family, or your doctor. Recent advances in computer vision, machine learning, and computational analysis now provide means to attack a critical goal of social neuroscience: how the social and affective system guides active information gathering in natural social contexts. We propose a novel natural approach to studying SV. This project will leverage a powerful technique to measure activity in the human brain: direct recording from electrodes implanted in the brains of surgical epilepsy patients. Surgical epilepsy patients who spend 1-2 weeks in the hospital while having their brain activity monitored afford the unique opportunity to record multiscale neural activity during natural interactions with friends, family, doctors, nurses, experimenters, etc. In addition, patients will play a social game to allow for the study of real world SV using a semi-controlled and repeatable task. Furthermore, participants will engage in more standard laboratory SV paradigms to assess how results from real world conditions compare to the results from traditional experiments. The neural recordings will be acquired simultaneously with video and audio monitoring, and eye tracking. State-of-the-art computer vision analysis will provide a continuous assessment of social cues (e.g., eye gaze and facial expression) from people with whom the patients are interacting. Computational analyses and mechanistic neural measurements will determine the correspondence between what is perceived by the patient, the neural coding, and the neurobiological implementation of that code. This approach will allow us to bridge across three critical levels of analysis required for understanding the SV information processing systems. We hypothesize that real world SV is an extended, iterative process that combines active, dynamic motor/attentional sampling strategies with prior and contextual information to actively plan information gathering, which both enables and constrains the flow of information through the system. Our results will have fundamental implications both for basic social neuroscience and for our understanding of how these processes may be altered in disorders of SV, such as mood and anxiety disorders and schizophrenia.

摘要/摘要 社会视觉神经科学的主导范式是呈现简化的刺激，通常是 由孤立的静态图像组成，没有上下文，被动地呈现给需要保持凝视的受试者 固定。虽然使用这种范式已经学到了很多东西，但它无法捕捉社会视觉（SV）如何的所有方面 作品。具体来说，主动观察者结合了运动信息收集行为（眼睛和头部运动） 在复杂、自然的多感官社会环境中对感知信息进行最佳采样。确实，有 很少有地方比 SV 的神经生物学研究更重要的是主动视觉。现实世界的 SV 是 原型主动传感过程，我们使用我们的经验，以及我们所有的传感和解释 在复杂和动态的环境中从其他人那里收集社会和情感信息的能力。简单地 简而言之，SV，正如通常所研究的那样，并不能充分模拟与朋友、家人或你的人的积极互动。 医生。计算机视觉、机器学习和计算分析的最新进展现在提供了手段 实现社会神经科学的一个关键目标：社会和情感系统如何引导主动信息 在自然的社会环境中聚集。我们提出了一种新的自然方法来研究 SV。 该项目将利用一种强大的技术来测量人脑的活动：直接记录 来自植入手术癫痫患者大脑中的电极。手术治疗癫痫患者花费1-2 在医院呆上几周，同时监测他们的大脑活动，这提供了独特的机会来记录 与朋友、家人、医生、护士、实验者等自然互动期间的多尺度神经活动。 此外，患者将玩社交游戏，以便使用半控制和半控制的方法来研究现实世界的 SV 可重复的任务。此外，参与者将参与更标准的实验室 SV 范式来评估如何 现实世界条件的结果与传统实验的结果进行比较。神经记录 将与视频和音频监控以及眼动追踪同时采集。最先进的计算机 视觉分析将提供对社交线索（例如眼神和面部表情）的持续评估 与患者互动的人。计算分析和机械神经测量 将确定患者的感知、神经编码和 该代码的神经生物学实现。这种方法将使我们能够跨越三个关键层面 理解 SV 信息处理系统所需的分析。我们假设现实世界的 SV 是一个扩展的迭代过程，它将主动、动态运动/注意力采样策略与先前的 和上下文信息来积极规划信息收集，这既可以促进也可以限制信息流 通过系统的信息。我们的结果将对基本社会产生根本性影响 神经科学，以及帮助我​​们了解这些过程在 SV 疾病（例如情绪）中如何改变 以及焦虑症和精神分裂症。