III:Medium:Physics-guided Machine Learning for Predicting Cell Trajectories, Shapes, and Interactions in Complex Dynamic Environments

III：中：物理引导机器学习，用于预测复杂动态环境中的细胞轨迹、形状和相互作用

基本信息

批准号：
2107332
负责人：
Anuj Karpatne
金额：
$ 100万
依托单位：
Virginia Polytechnic Institute and State University
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2021
资助国家：
美国
起止时间：
2021-10-01 至 2025-09-30
项目状态：
未结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2107332&HistoricalAwards=false
关键词：
III Medium Physics guided Machine

项目摘要

As advances in deep learning continue to revolutionize the field of computer vision, it is now possible for machine learning methods to predict the future trajectory and behavior of moving objects in benchmark problems such as pedestrian and vehicle tracking. Despite these developments, current standards in deep learning for predicting future trajectories of objects mostly assume the background to be static and the shapes of the objects to be invariant to motion. However, in many real-world applications, we routinely encounter problems where the background environment is constantly changing its structure, which in-turn directly affects changes in shape, appearance, and future trajectory of the moving objects. For example, in the area of mechanobiology—the field of study of movements of living cells—cells undergo massive transformations in their shape, size, and trajectory as they move across fibrous environments in the human body, continuously tugging or pushing on the background fibers and remodeling the background environment in the process. This project aims to develop novel machine learning methods to study the interplay between changes in cell shapes and background environments using microscopy imaging data and scientific knowledge of the physics of forces exerted by the cells on the background environments. Our ultimate objective is to discover the rules of cell behavior under varying background configurations and use these rules to predict future movements of cells in a number of scientific and societally relevant applications such as the study of embryo development, wound closure, immune response, and cancer metastasis. One of the long-standing goals of artificial intelligence has been to teach machines how to predict or forecast the future. With advances in deep learning, it is now possible for machine learning (ML) frameworks to make predictions in several computer vision applications. We ask the question: can deep learning methods extract the rules of motion of dynamic “shape-shifting” objects—that are constantly adapting their appearance in relation to their environment—and use these rules to predict their future behavior? We investigate this question in the context of a motivation application in mechanobiology to predict and explain how cells move, interact with each other, remodel their environment, and adapt their appearance with changing physiological environments inside our body. Despite the success of deep learning in predicting human motion and vehicle trajectories, fundamental gaps remain in the ability of these methods to predict the dynamics of cell motion in complex realistic environments. This is primarily due to the highly dynamic nature of cell shapes that undergo limitless transformations as they sense and react to their environment during motion. In addition, the dynamics of cell motion is constrained by the physics of forces exerted by the cells on the background environment, as well as the complex nature of cell-cell interactions. The vision of this project is to develop a novel physics-guided machine learning (PGML) framework to predict the motion of shape-shifting objects in dynamic physical environments. Our framework fully leverages the principles of “convergence research” by integrating data, knowledge, and methodologies from three different disciplines: machine learning, experimental cell imaging, and computational modeling. The ultimate goal of our project is to catalyze the discovery of new “rules of cell behavior” by analyzing explainable theories produced by our PGML framework in the context of mechanobiology.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

随着深度学习的进步继续彻底改变计算机视觉领域，机器学习方法现在可以在行人和车辆跟踪等基准问题中预测移动物体的未来轨迹和行为，尽管取得了这些进展，但当前的标准仍在深度学习中。用于预测物体未来轨迹的学习大多假设背景是静态的，而物体的形状对于运动是不变的。然而，在许多现实世界的应用中，我们经常遇到背景环境不断改变其结构的问题。反过来又直接影响形状的变化，例如，在机械生物学领域——研究活细胞运动的领域——细胞在穿过纤维环境时，其形状、大小和轨迹会发生巨大的变化。该项目旨在开发新颖的机器学习方法，利用显微镜成像数据和科学知识来研究细胞形状的变化和背景环境之间的相互作用。力的物理学我们的最终目标是发现不同背景配置下细胞行为的规则，并利用这些规则来预测细胞在许多科学和社会相关应用（例如胚胎发育研究）中的未来运动。、伤口闭合、免疫反应和癌症转移是人工智能的长期目标之一是教会机器如何预测或预测未来。随着深度学习的进步，机器学习 (ML) 现在已成为可能。在多种计算机视觉应用中进行预测的框架。提出一个问题：深度学习方法能否提取动态“变形”物体的运动规则（这些物体不断地根据环境调整其外观）并使用这些规则来预测它们未来的行为？尽管深度学习在预测人体运动和车辆轨迹方面取得了成功，但在机械生物学中应用动机来预测和解释细胞如何移动、相互作用、重塑其环境以及如何根据体内不断变化的生理环境调整其外观。能力方面仍存在根本差距这些方法可以预测复杂现实环境中细胞运动的动力学，这主要是由于细胞形状的高度动态性质，它们在运动过程中感知环境并对其做出反应时会发生无限的变化。受到细胞对背景环境施加的物理力以及细胞与细胞相互作用的复杂性的限制，该项目的愿景是开发一种新颖的物理引导机器学习（PGML）框架来预测运动。动态物理环境中变形物体的研究。该框架通过整合来自三个不同学科的数据、知识和方法，充分利用“融合研究”的原则：机器学习、实验细胞成像和计算建模，我们项目的最终目标是促进新“规则”的发现。通过分析我们的 PGML 框架在机械生物学背景下产生的可解释理论，“细胞行为”。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

期刊论文数量（8）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

MEMTRACK: A Deep Learning-Based Approach to Microrobot Tracking in Dense and Low-Contrast Environments

MEMTRACK：一种基于深度学习的方法，用于在密集和低对比度环境中跟踪微型机器人

DOI：
10.48550/arxiv.2310.09441
发表时间：
2023-10-13
期刊：
ArXiv
影响因子：
0
作者：
Medha Sawhney;Bhas Karmarkar;E. Leaman;Arka Daw;A. Karpatne;B. Behkam
通讯作者：
B. Behkam

Mitigating Propagation Failures in Physics-Informed Neural Networks Using Retain-Resample-Release (R3) Sampling

使用保留-重采样-释放 (R3) 采样减轻物理信息神经网络中的传播失败

DOI：
发表时间：
2023-07
期刊：
Proceedings of Machine Learning Research
影响因子：
0
作者：
Daw, Arka and;Bu, Jie;Wang, Sifan;Perdikaris, Paris;Karpatne, Anuj
通讯作者：
Karpatne, Anuj

Experimental and theoretical model for the origin of coiling of cellular protrusions around fibers

纤维周围细胞突起卷绕起源的实验和理论模型

DOI：
10.1038/s41467-023-41273-y
发表时间：
2023-09-12
期刊：
Nature Communications
影响因子：
16.6
作者：
Raj Kumar Sadhu;Christian Hern;ez;ez;Yael Eshed Eisenbach;S. Penič;Lixia Zhang;Harshad D. Vishwasrao;B. Behkam;K. Konstantopoulos;H. Shroff;A. Iglič;E. Peles;A. Nain;N. Gov
通讯作者：
N. Gov

Mitigating Propagation Failures in Physics-Informed Neural Networks Using Retain-Resample-Release (R3) Sampling

使用保留-重采样-释放 (R3) 采样减轻物理信息神经网络中的传播失败

DOI：
发表时间：
2023-07
期刊：
Proceedings of the 40th International Conference on Machine Learning
影响因子：
0
作者：
Daw, Arka;Bu, Jie;Wang, Sifan;Perdikaris, Paris;Karpatne, Anuj
通讯作者：
Karpatne, Anuj

Sculpting Rupture‐Free Nuclear Shapes in Fibrous Environments

在纤维环境中塑造无破裂的核形状

DOI：
10.1002/advs.202203011
发表时间：
2022-07
期刊：
Advanced Science
影响因子：
15.1
作者：
Jana, Aniket;Tran, Avery;Gill, Amritpal;Kiepas, Alexander;Kapania, Rakesh K.;Konstantopoulos, Konstantinos;Nain, Amrinder S.
通讯作者：
Nain, Amrinder S.