CHS: Small: High Resolution Motion Capture

CHS：小：高分辨率运动捕捉

• 批准号: 2008564
• 负责人: Yin Yang
• 金额: $ 49.97万
• 依托单位: University of Utah
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2008564&HistoricalAwards=false
• 关键词: CHS; Small; Resolution; Motion; Capture

This project studies a new type of full-body motion capture technology which has been enabled by recent advances in deep learning and high-resolution digital cameras. Unlike classical motion capture systems which rely on suits with small attached spheres that serve as markers, this project introduces a new type of suit using a special printed pattern instead of any attachments. This pattern will contain a new type of markers with two distinct advantages: (1) the ability to automatically detect which marker is which, and (2) a significantly more dense set of markers than previous systems. The proposed approach is fully passive and therefore easy to use, relying only (a) indoor lighting or natural daylight and (b) a suit made of elastic fabric with a special printed pattern, but without any wires or batteries. The only required electronics are standard digital cameras, ranging from just a few cameras to massive multi-camera systems. This flexibility will allow us to support applications on various scales, from individual researchers or makers to large institutions or production studios. New technology for full-body capture, featuring higher accuracy while being easy to use, has the potential to impact research and clinical studies of human motion, e.g., in orthopedics, sports medicine, rehabilitation, physical therapy and ergonomics. High-quality human motion data can also facilitate better virtual or augmented reality systems and applications. By combining computer science and human motion, motion capture systems enables unique educational and outreach opportunities involving activities popular among young people, such as sports and gymnastics. The idea of using a new type of motion capture suit with texture-based markers recognized using artificial neural networks has not been explored before and opens up many interesting research questions such as "Which types of markers and suit textures will lead to the best detection and labeling results, despite large elastic distortions induced by the motion of the skin?" The proposed computing methodology requires training and validation of neural networks and contributes to research on the following topics: (1) synthetic data generation, (2) automated data augmentation, (3) confidence calibration of neural networks, in particular teaching neural networks to quantify the risk of errors in their own predictions. To further improve robustness and accuracy, the probabilistic output of neural networks will be combined with priors, such as a 3D deformable shape model; this can be done in a principled way via Bayesian inference, which leads to research problems combining continuous and discrete optimization. Finally, the high-resolution data produced by the envisioned system motivates research on improving the anatomical realism of deformable shape models, in particular data-driven modeling of joint kinematics and muscle activations. The envisioned full-body capture system will be designed to be easy to replicate and deploy at various institutions, clinics or studios. This project will share research results through common open source codebase, facilitating collaboration and data sharing.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目研究一种新型全身动作捕捉技术，该技术是由深度学习和高分辨率数码相机的最新进展实现的。与传统的动作捕捉系统依赖于带有小球体作为标记的套装不同，该项目引入了一种新型套装，使用特殊的印刷图案而不是任何附件。该模式将包含一种新型标记，具有两个明显的优点：（1）能够自动检测哪个标记是哪个，以及（2）比以前的系统明显更密集的标记集。所提出的方法是完全被动的，因此易于使用，仅依赖于（a）室内照明或自然日光和（b）由带有特殊印刷图案的弹性织物制成的套装，但没有任何电线或电池。唯一需要的电子设备是标准数码相机，范围从几个相机到大型多相机系统。这种灵活性将使我们能够支持各种规模的应用程序，从个人研究人员或制造商到大型机构或制作工作室。全身捕捉新技术具有更高的准确度且易于使用，有可能影响人体运动的研究和临床研究，例如骨科、运动医学、康复、物理治疗和人体工程学。高质量的人体运动数据还可以促进更好的虚拟或增强现实系统和应用程序。通过将计算机科学和人体动作相结合，动作捕捉系统提供了独特的教育和推广机会，涉及年轻人流行的活动，例如体育和体操。使用带有使用人工神经网络识别的基于纹理的标记的新型运动捕捉套装的想法以前从未被探索过，并且提出了许多有趣的研究问题，例如“哪种类型的标记和套装纹理将导致最佳的检测和识别”。尽管皮肤运动引起较大的弹性变形，但标签结果如何？”所提出的计算方法需要对神经网络进行训练和验证，并有助于以下主题的研究：（1）合成数据生成，（2）自动数据增强，（3）神经网络的置信度校准，特别是教导神经网络进行量化他们自己的预测出现错误的风险。为了进一步提高鲁棒性和准确性，神经网络的概率输出将与先验相结合，例如3D可变形形状模型；这可以通过贝叶斯推理以原则性的方式完成，这导致了连续和离散优化相结合的研究问题。最后，设想的系统产生的高分辨率数据激发了关于提高可变形形状模型的解剖真实性的研究，特别是关节运动学和肌肉激活的数据驱动建模。设想的全身捕捉系统将被设计为易于在各种机构、诊所或工作室复制和部署。该项目将通过通用开源代码库分享研究成果，促进协作和数据共享。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。