Collaborative Research: EAGER: Deep Learning-based Multimodal Analysis of Sleep

合作研究：EAGER：基于深度学习的睡眠多模态分析

• 批准号: 2334665
• 负责人: Ping Chen
• 金额: $ 12.5万
• 依托单位: University of Massachusetts Boston
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-10-01 至 2025-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2334665&HistoricalAwards=false
• 关键词: Collaborative; Research; EAGER; Deep; Learning

Sleep is one of the most fundamental behaviors for animals and humans, and understanding group sleep will provide key insights into neuroscience and social behavior and interactions. To overcome limitations that stem from single modality animal behavior platforms, the project will develop a multimodal machine learning method to simultaneously monitor and process Electroencephalogram (EEG) data and animal behavior data to systematically study group behavior, especially sleep, and to annotate animal social movements/behavior. The outcomes from the project will potentially provide a powerful toolkit based on deep learning to make sense of complex animal behavior and EEG activity pattern for mechanistic exploration. Subproblems from this project will be developed into course materials and will be capstone projects or directed study for undergraduate students.The project will process multiple data modalities and group activities involving multiple entities from multiple data sources through a multi-modal machine learning framework enabling the extraction and aggregation of the most pertinent information. A “dictionary” of movements at the semantic level will be developed for learning and processing of long video and EEG data, which is a significant challenge for current state-of-the-art self-attention transformer models. Additionally, to incorporate group interactions, transformer models for dialogue modeling will be developed. The proposed simultaneous EEG and behavior study will provide biological underpinnings of group sleep, leading to insights into brain electrical signaling and behavioral outputs - how the brain marshals its signaling units to generate behaviors.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

睡眠是动物和人类最基本的行为之一，了解群体睡眠将为神经科学、社会行为和互动提供重要见解。为了克服单一模式动物行为平台的限制，该项目将开发一种多模式机器学习方法。同时监测和处理脑电图（EEG）数据和动物行为数据，系统地研究群体行为，特别是睡眠，并注释动物的社会运动/行为。该项目的结果将有可能提供一个基于深度学习的强大工具包，以使其有意义。复杂动物的该项目的子问题将被开发成课程材料，并将成为本科生的顶点项目或定向研究。该项目将通过一个数据源处理涉及多个实体的多种数据模式和小组活动。多模态机器学习框架能够提取和聚合最相关的信息，将开发语义级别的运动“字典”，用于学习和处理长视频和脑电图数据，这对当前来说是一个重大挑战。此外，为了整合群体互动，将开发用于对话建模的变压器模型，所提出的同步脑电图和行为研究将为群体睡眠提供生物学基础，从而深入了解脑电。信号和行为输出——大脑如何整理其信号单元以产生行为。这反映了 NSF 奖项的法定使命，并且通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。