CHS: Small: High Resolution Motion Capture

CHS：小：高分辨率运动捕捉

• 批准号: 2008564
• 负责人: Yin Yang
• 金额: $ 49.97万
• 依托单位: University of Utah
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2008564&HistoricalAwards=false
• 关键词: CHS; Small; Resolution; Motion; Capture

This project studies a new type of full-body motion capture technology which has been enabled by recent advances in deep learning and high-resolution digital cameras. Unlike classical motion capture systems which rely on suits with small attached spheres that serve as markers, this project introduces a new type of suit using a special printed pattern instead of any attachments. This pattern will contain a new type of markers with two distinct advantages: (1) the ability to automatically detect which marker is which, and (2) a significantly more dense set of markers than previous systems. The proposed approach is fully passive and therefore easy to use, relying only (a) indoor lighting or natural daylight and (b) a suit made of elastic fabric with a special printed pattern, but without any wires or batteries. The only required electronics are standard digital cameras, ranging from just a few cameras to massive multi-camera systems. This flexibility will allow us to support applications on various scales, from individual researchers or makers to large institutions or production studios. New technology for full-body capture, featuring higher accuracy while being easy to use, has the potential to impact research and clinical studies of human motion, e.g., in orthopedics, sports medicine, rehabilitation, physical therapy and ergonomics. High-quality human motion data can also facilitate better virtual or augmented reality systems and applications. By combining computer science and human motion, motion capture systems enables unique educational and outreach opportunities involving activities popular among young people, such as sports and gymnastics. The idea of using a new type of motion capture suit with texture-based markers recognized using artificial neural networks has not been explored before and opens up many interesting research questions such as "Which types of markers and suit textures will lead to the best detection and labeling results, despite large elastic distortions induced by the motion of the skin?" The proposed computing methodology requires training and validation of neural networks and contributes to research on the following topics: (1) synthetic data generation, (2) automated data augmentation, (3) confidence calibration of neural networks, in particular teaching neural networks to quantify the risk of errors in their own predictions. To further improve robustness and accuracy, the probabilistic output of neural networks will be combined with priors, such as a 3D deformable shape model; this can be done in a principled way via Bayesian inference, which leads to research problems combining continuous and discrete optimization. Finally, the high-resolution data produced by the envisioned system motivates research on improving the anatomical realism of deformable shape models, in particular data-driven modeling of joint kinematics and muscle activations. The envisioned full-body capture system will be designed to be easy to replicate and deploy at various institutions, clinics or studios. This project will share research results through common open source codebase, facilitating collaboration and data sharing.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目研究了一种新型的全身运动捕获技术，该技术已通过深度学习和高分辨率数码相机的最新进展来实现。与经典的运动捕获系统不同，这些系统依赖于具有小的附件球体的西装，该项目使用特殊的印刷图案而不是任何附件引入了一种新型的西装。该模式将包含具有两个不同优势的新类型标记：（1）自动检测哪个标记的能力，以及（2）比以前的系统更密集的标记集。所提出的方法是完全被动的，因此易于使用，仅依靠（a）室内照明或自然日光以及（b）由具有特殊印刷图案的弹性织物制成的西装，但没有任何电线或电池。唯一必需的电子设备是标准数码摄像机，范围从几个相机到大型多摄像机系统。这种灵活性将使我们能够在各种规模上支持应用程序，从个人研究人员或制造商到大型机构或生产工作室。全身捕获的新技术具有更高的精度，同时易于使用，具有影响人类运动的研究和临床研究，例如骨科，运动医学，康复，物理治疗和人体工程学。高质量的人类运动数据还可以促进更好的虚拟或增强现实系统和应用。通过结合计算机科学和人类运动，运动捕获系统可以实现独特的教育和外展机会，涉及在运动和体操等年轻人中流行的活动。以前尚未探索使用一种使用人工神经网络认可的基于纹理的标记的新型运动捕捉套装的想法，但仍未探讨许多有趣的研究问题，例如“尽管皮肤运动的运动引起了巨大的弹性扭曲，但“哪种类型的标记和西装纹理将导致最佳的检测和标记结果？”提出的计算方法需要对神经网络进行培训和验证，并有助于研究以下主题：（1）合成数据生成，（2）自动数据增强，（3）神经网络的置信度校准，特别是教学神经网络以量化自己预测的错误风险。为了进一步提高鲁棒性和准确性，神经网络的概率输出将与先验（例如3D可变形形状模型）结合使用；这可以通过贝叶斯推论以原则性的方式完成，这导致了结合持续和离散优化的研究问题。最后，设想系统产生的高分辨率数据激发了研究改善可变形形状模型的解剖现实主义，特别是关节运动学和肌肉激活的数据驱动的建模。设想的全身捕获系统将易于复制，并在各个机构，诊所或工作室部署。该项目将通过常见的开源代码库分享研究结果，促进协作和数据共享。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子优点和更广泛的影响审查标准通过评估来获得支持的。