CAREER: Deep Learning Empowered Nonlinear Deformable Model

职业：深度学习赋能非线性变形模型

• 批准号: 2301040
• 负责人: Yin Yang
• 金额: $ 55万
• 依托单位: University of Utah
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-10-01 至 2025-04-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2301040&HistoricalAwards=false
• 关键词: CAREER; Deep; Learning; Empowered; Nonlinear

Everything in the world deforms, so modeling high-quality deformations becomes a core algorithmic ingredient for serious and realism-driven visual applications such as high-fidelity animation, virtual reality, medical data analysis, surgical simulation, and digital fabrication/prototyping, to name just a few. While deformation has been studied for decades, deformable simulation is notorious for its costly computation. With the rapid development of sophisticated sensing devices and acquisition techniques, the complexities, scales and dimensionalities of the data have grown exponentially, and large-scale geometries are becoming ubiquitous in modern 3D data processing. Even with state-of-the-art hardware, a massive deformable simulation can still take hours, days, or even weeks. In this era of data explosion, increasing demands on both computing efficiency and simulation realism impose unprecedented challenges on this classic computing problem, so game-changing algorithmic techniques for large-scale, complex, and nonlinear deformable models are needed to empower future graphics applications. If successful, this project will not only expand the frontier of physics-based simulation technologies, but also profoundly inspire broader computing communities beyond graphics and enable a variety of applications. During a deformable simulation, a nonlinear system needs to be repetitively solved in order to track the continuous shape evolution of the deforming body. A deformable object with complex geometry could house a large number of unknown degrees of freedom, and the resulting high-dimensional integration becomes prohibitive. To overcome this problem, this project will develop a re-branded deformable model which systematically integrates advanced simulation techniques and deep learning (DL) tools, specifically deep neural networks (DNNs). The hypothesis is that digital simulation provides us nearly unlimited noise-free training data, which should be fully exploited and leveraged to benefit unseen yet difficult simulation or computing challenges. Unlike existing data-driven methods that interpret the data with a closed-form formulation (e.g., using a convex interpolation), DNNs provide a universal mechanism to extract intrinsic features hidden behind the raw data in an end-to-end manner, and have already demonstrated significant outcomes in many long-standing computer vision problems like object detection, classification, and annotation. However, harnessing DL in physics-based simulation is not easy. While in theory one may still encode all of these parameters using a very high-dimensional input vector, the corresponding network would be extremely large and complex. Even if we manage to collect sufficient training data to optimize this net, a single forward pass of it may be slower than a conventional simulator, making DL completely unprofitable. In this project, we will thoroughly investigate those grand technical challenges, forge a collection of data structures and algorithmic techniques for the data-driven deformable simulation, and thereby pave the way for DL-based physics simulation to next-generation computer graphics.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

世界上的所有事物变形，因此，建模高质量的变形成为严肃和现实驱动的视觉应用，例如高保真动画，虚拟现实，医学数据分析，手术模拟和数字制造/原型型等严重和现实驱动的视觉应用的核心算法成分。尽管已研究变形数十年，但可变形模拟以其昂贵的计算而臭名昭著。随着复杂的传感设备和采集技术的快速发展，数据的复杂性，量表和维度呈指数增长，并且在现代3D数据处理中，大规模的几何形状变得无处不在。即使使用最先进的硬件，大量可变形的模拟仍然可能需要数小时，数天甚至几周。在这个数据爆炸时代，对计算效率和模拟现实主义的需求不断提高，对这个经典的计算问题构成了前所未有的挑战，因此需要针对大规模，复杂和非线性可变形模型进行游戏改变的算法技术来增强未来图形应用。 如果成功的话，该项目不仅将扩大基于物理的模拟技术的前沿，而且还会深刻激发更广泛的计算社区，超越图形，并启用各种应用。在可变形的模拟过程中，需要重复求解非线性系统，以跟踪变形体的连续形状演变。具有复杂几何形状的可变形物体可以容纳大量未知的自由度，并且所得的高维整合变得过于敏锐。为了克服这个问题，该项目将开发一个重新品牌的可变形模型，该模型会系统地集成高级仿真技术和深度学习工具（DL）工具，特别是深神经网络（DNNS）。假设是，数字仿真为我们提供了几乎无限的无噪声训练数据，这些数据应充分利用和利用，以使看不见但困难的模拟或计算挑战受益。与现有的数据驱动方法不同，这些方法以封闭式配方（例如，使用凸插值）来解释数据，DNN提供了一种通用机制，可以以端到端的方式提取原始数据隐藏在原始数据后面的固有功能，并且已经在许多长期存在的计算机视觉问题（例如对象检测，分类，分类和注释）中表现出重大的结果。但是，在基于物理的模拟中利用DL并不容易。从理论上讲，人们仍然可以使用非常高维的输入向量编码所有这些参数，但相应的网络将非常大且复杂。即使我们设法收集足够的培训数据来优化该网络，它的单个前向通行证也可能比常规模拟器慢，这使DL完全无利可图。在这个项目中，我们将彻底研究这些宏伟的技术挑战，为数据驱动的可变形模拟制定了一系列数据结构和算法技术，从而为下一代计算机图形学铺平了道路，为NSF的法定任务和评估者的支持构成了概念的范围，从而为下一代计算机图形提供了依据。