cloudSLEAP: Maximizing accessibility to deep learning-based motion capture

cloudSLEAP：最大限度地提高基于深度学习的动作捕捉的可访问性

• 批准号: 10643661
• 负责人: Talmo D. Pereira
• 金额: $ 262.32万
• 依托单位: SALK INSTITUTE FOR BIOLOGICAL STUDIES
• 依托单位国家: 美国
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-09-01 至 2026-08-31
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/10643661
• 关键词: Acceleration; Address; Adoption; Animal Behavior; Animal Model; Animals; BRAIN initiative; Behavior; Behavioral; Body part; Brain; Cloud Computing; Cloud Service; Collaborations; Communication; Communities; Complex; Computer Hardware; Computer software; Data; Data Set; Dedications; Democracy; Dependence; Development; Documentation; Ecosystem; Educational Activities; Ensure; Environment; Equity; Ethology; Event; Future; Goals; Industry; Informatics; Infrastructure; Institution; Internet; Investments; Label; Libraries; Modeling; Modernization; Motion; Neurosciences; Occupations; Online Systems; Outcome; Output; Persons; Posture; Qualifying; Research; Research Personnel; Resolution; Resources; Running; Services; Software Engineering; Standardization; System; Technical Expertise; Technology; Testing; Training; Training and Education; Training and Infrastructure; Visualization; Work; cloud based; computer infrastructure; computerized tools; computing resources; cost; cost effective; data archive; data format; data repository; data standards; deep learning; experience; gigabyte; graphical user interface; learning strategy; light weight; meetings; next generation; open source; public repository; recruit; tool; usability; virtual; virtual machine; web platform

cloudSLEAP – PROJECT SUMMARY/ABSTRACT Understanding how the brain produces complex behavior is a central goal of neuroscience, but quantifying behavior is technically challenging, particularly in unrestrained and naturalistic settings. Tools that are able to overcome these limitations leverage deep learning to achieve robust markerless motion capture, enabling characterization of behavior through precise positional tracking of body parts from standard videos of behavior. Unfortunately, like most deep learning systems, existing pose tracking software requires technical expertise to manage the complex software dependencies required for deep learning, and investment in expensive computational hardware (GPUs), both of which curtail equitable access to this technology. This project proposes cloudSLEAP, a platform that builds on the widely used multi-animal pose tracking software SLEAP to address these barriers by providing the infrastructure necessary to run the entire pose tracking workflow through cloud-based systems. This platform enables annotation, visualization and sharing pose tracking datasets directly from the browser, eliminating the need for installation and management of desktop-based software. cloudSLEAP will support data formats from all currently existing tools for pose tracking, and will be integrated with existing data standards and repositories such as NeurodataWithoutBorders and DANDI to ensure that all outputs of cloudSLEAP are standardized and FAIR-compliant. Users will be able to use cloudSLEAP to train pose tracking models on their own data through a cloud-based job orchestration system, eliminating the complexity of deep learning library dependencies. Leveraging the highly efficient model configurations provided by SLEAP, cloudSLEAP will provide users with free computational resources on the cloud to train pose models. This capability effectively eliminates the need for investment in local GPU hardware, thereby removing the single biggest barrier to entry for researchers from under-resourced institutions. The entire platform will be developed as open-source software on public repositories from the start, and all data used for testing and development will be freely available. A core goal for this project is to ensure that cloudSLEAP maximizes accessibility to behavior quantification technology to the widest range of practitioners. To this end, the first aim of this proposal will be to establish a broad-based community of beta testers that represents the diversity of institutions in the BRAIN Initiative and wider neuroscience community. Beta testers will be invited to collaborate throughout development via regular virtual Town Hall meetings, in-person events, direct communication channels and open discussion forums. These efforts will additionally produce documentation and didactic materials that will be used for training and education activities. By ensuring that diverse perspectives are included from the very onset of the project, cloudSLEAP will enable truly equitable access and dissemination of a crucial part of the modern neuroscience toolkit.

cloudSLEAP – 项目摘要/摘要 了解大脑如何产生复杂的行为是神经科学的核心目标，但量化 行为在技术上具有挑战性，特别是在不受限制和自然的环境中。 克服这些限制，利用深度学习实现强大的无标记动作捕捉，从而实现 通过从标准行为视频中精确跟踪身体部位的位置来表征行为。 不幸的是，与大多数深度学习系统一样，现有的姿势跟踪软件需要技术专业知识才能 管理深度学习所需的复杂软件依赖项以及昂贵的投资 计算硬件（GPU），两者都限制了该项目的公平使用。 提出了cloudSLEAP，这是一个基于广泛使用的多动物姿势跟踪软件SLEAP构建的平台 通过提供运行整个姿势跟踪工作流程所需的基础设施来解决这些障碍 通过基于云的系统，该平台可以实现注释、可视化和共享姿势跟踪。 直接从浏览器获取数据集，无需安装和管理基于桌面的 cloudSLEAP 软件将支持当前所有现有姿势跟踪工具的数据格式，并将 与现有数据标准和存储库（例如 NeurodataWithoutBorders 和 DANDI）集成 确保cloudSLEAP的所有输出都是标准化的并且符合FAIR标准，用户将能够使用。 cloudSLEAP 通过基于云的作业编排系统根据自己的数据训练姿势跟踪模型， 利用高效模型 SLEAP提供的配置，cloudSLEAP将为用户提供免费的计算资源 此功能有效地消除了对本地 GPU 的投资需求。 硬件，从而消除了资源不足的研究人员进入的最大障碍 整个平台将从一开始就作为公共存储库上的开源软件进行开发， 用于测试和开发的所有数据都将免费提供，该项目的核心目标是确保。 cloudSLEAP 最大限度地提高了最广泛的行为量化技术的可访问性 为此，该提案的首要目标是建立一个基础广泛的 beta 社区。 代表 BRAIN Initiative 和更广泛的神经科学界机构多样性的测试人员。 Beta 测试人员将被邀请通过定期的虚拟市政厅会议在整个开发过程中进行协作， 此外，还包括现场活动、直接沟通渠道和开放讨论论坛。 制作用于培训和教育活动的文档和教学材料。 确保从项目一开始就包含不同的观点，cloudSLEAP 将真正实现 公平获取和传播现代神经科学工具包的重要组成部分。

项目成果

Fast and Efficient Root Phenotyping via Pose Estimation.

通过姿势估计进行快速高效的根表型分析。

• 发表时间: 2024
• 作者: Berrigan, Elizabeth M;Wang, Lin;Carrillo, Hannah;Echegoyen, Kimberly;Kappes, Mikayla;Torres, Jorge;Ai;McCoy, Erica;Shane, Emily;Copeland, Charles D;Ragel, Lauren;Georgousakis, Charidimos;Lee, Sanghwa;Reynolds, Dawn;Talgo, Ave
• 通讯作者: Talgo, Ave

Fast and efficient root phenotyping via pose estimation.

通过姿态估计快速有效地进行根表型分析。

• 发表时间: 2023-11-21
• 作者: Berrigan, Elizabeth M;Wang, Lin;Carrillo, Hannah;Echegoyen, Kimberly;Kappes, Mikayla;Torres, Jorge;Ai;McCoy, Erica;Shane, Emily;Copeland, Charles D;Ragel, Lauren;Georgousakis, Charidimos;Lee, Sanghwa;Reynolds, Dawn;Talgo, Ave
• 通讯作者: Talgo, Ave