CAREER: Multimodal Brain and Body Music Interfaces to Promote Entrainment, Connection, and Creative Science Education

职业：多模式大脑和身体音乐界面促进夹带、联系和创造性科学教育

• 批准号: 2142959
• 负责人: Grace Leslie
• 金额: $ 48.99万
• 依托单位: Georgia Tech Research Corporation
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-06-15 至 2023-04-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2142959&HistoricalAwards=false
• 关键词: CAREER; Multimodal; Brain; Body; Music

This award is funded in part under the American Rescue Plan Act of 2021 (Public Law 117-2). Entrainment is a process in which people’s natural brain and body rhythms synchronize, through stimuli such as music, which may create feelings of connection and well-being. This project addresses entrainment by building multimodal signal mapping interfaces that mediate interpersonal connections by deriving music from brain and body rhythms. The investigator will integrate sensor hardware and signal processing software to stream live brain and body data, perform calculations to extract signal characteristics, and use this to drive sound synthesis. A series of music cognition and listening experiments study physiological, behavioral, and affective entrainment phenomena, which are expected to result, from a series of multimodal brain music interfaces. A use-case study, developed in consultation with doctors, connects mothers and infants, physically separated by distance, using the multimodal entrainment interface. Mother and infant hear music derived from each other’s heartbeats and breathing. This study investigates the entrainment created in their body rhythms, and maps health and well-being effects of the virtual connection environment. For researchers, doctors, and caretakers, multimodal brain music interfaces have the potential to expand our scientific understanding of music’s beneficial effects on the brain and body, which may lead to new health and well-being interventions for adults, children, and infants. This project will result in an open-source tool kit of accessible technologies and STEM learning modules to inspire educators and students to develop projects that further our understanding of brain and body signals. These learning modules will be integrated into a summer research experience--involving high school students and their teachers--in which authentic learning encourages students’ training in the scientific method through their natural interest in music. This project develops and evaluates an interface with new multimodal signal mapping technologies that translate neurophysiological signals (e.g., EEG, ECG, EDA, respiration) into musical sound to promote biological, behavioral, and affective synchrony between individuals and computers by: (1) engineering sonification techniques that perform real-time signal processing and algorithmic music generation for transforming physiological signals into music; (2) investigating the neuropsychological mechanisms that govern auditory neurostimulation and physiological entrainment by designing new rhythmic auditory neurophysiological sonification stimuli and measuring how the human body responds; and (3) designing and evaluating a use case that involves co-generating music for infants and their mothers with each other’s physiological data. Quantitative data will address synchronies in physiology, protocol analysis of video will address behavioral synchronies, and qualitative data will address experiences. These research activities will contribute to an overarching goal of discovering how using computing to pair music and physiology can function as a significant information channel in human-centered computing. One expected use of this channel is to promote human connection and well-being through entrainment.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该奖项是根据2021年《美国救援计划法》（第117-2）的部分资助。夹带是人们通过音乐（例如音乐）的刺激来同步人们的自然大脑和身体节奏的过程，这可能会产生联系和幸福感。该项目地址通过构建多模式信号映射界面，通过从大脑和身体节奏中派生音乐来介导人际交往。研究者将集成传感器硬件和信号处理软件以流式传输活脑和身体数据，执行计算以提取信号特征，并使用它来驱动声音合成。一系列音乐的认知和听力实验研究生理，行为和情感入口现象，这些现象有望从一系列多模式的脑音乐接口中产生。一项使用多模式入口界面与医生咨询的用例研究，与医生咨询，将母亲和婴儿与距离进行物理分离。母亲和婴儿听到彼此的心跳和呼吸中衍生出的音乐。这项研究调查了其身体节奏中产生的入口，并绘制虚拟连接环境的健康和福祉效果。对于研究人员，医生和看护人，多模式的脑音乐界面有可能扩大我们对音乐对大脑和身体的有益影响的科学理解，这可能会导致成人，儿童和婴儿的新健康和福祉干预措施。该项目将为可访问的技术和STEM学习模块提供开源工具套件，以激发教育工作者和学生开发对我们对大脑和身体信号的理解的项目。这些学习模块将纳入夏季研究经验（涉及高中生及其老师），这在真实的学习中鼓励学生通过对音乐的自然兴趣接受科学方法的培训。该项目开发并评估了与新的多模式信号映射技术的界面，该技术将神经生理学信号（例如EEG，EEG，ECG，ECG，EDA，呼吸）转化为音乐声音，以促进个人和计算机之间的生物学，行为和情感同步：（1）通过以下方式进行：（1）执行实时信号的音乐，以进行实时信号的音乐，以进行实时信号的启发，以实现音乐的启发，以进行实时的启发，以实现音乐的过程，以实现音乐的过程，以实现音乐的过程，以实现型号，以实现音乐性的过程，以实现音乐的过程，以实现音乐性的过程，以实现音乐性的过程，以实现音乐性的过程，以实现音乐性的过程，以实现型号，以实现型号，以实现音乐性的过程，以实现音乐的过程，以实现型号，以实现音乐的过程，以实现音乐的过程，以实现音乐的启发，以实现音乐性的过程; （2）研究神经心理学机制，这些机制通过设计新的有节奏的听觉神经生理超声刺激并测量人体的反应来控制听觉神经刺激和生理入口； （3）设计和评估用例，该用例涉及为婴儿及其母亲与彼此的物理数据共同创造音乐。定量数据将解决生理学的同步，视频协议分析将解决行为同步，定性数据将解决经验。这些研究活动将有助于发现如何使用计算来配对音乐和生理学如何充当以人为本的计算中的重要信息渠道。该渠道的一种预期用途是通过入口来促进人际关系和福祉。该奖项反映了NSF的法定任务，并通过使用基金会的知识分子优点和更广泛的影响审查标准来评估被认为是宝贵的支持。