EAGER: Deep Causal Representation Learning for Generalizable Visual Understanding

EAGER：用于泛化视觉理解的深度因果表示学习

• 批准号: 2236026
• 负责人: Qiang Ji
• 金额: $ 30万
• 依托单位: Rensselaer Polytechnic Institute
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-09-01 至 2024-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2236026&HistoricalAwards=false
• 关键词: EAGER; Deep; Causal; Representation; Learning

The recent developments in deep learning have led to remarkable progress in computer vison and other fields. Current computer vision systems, however, do not perform well in a new environment different from its training environment. This problem, if left unaddressed, could lead to predictive errors and hence significant consequences for life-critical and high-stakes applications such as autonomous driving, robotics, and manufacturing. To address this challenge, this project introduces a novel deep causal learning framework using latent features. Unlike most features that are observable, latent features are hidden. For example, in an object recognition task, latent factors can be vectors describing the scale and poses of objects extracted from the image’s pixels and labels of every training image. The framework leverages deep learning to learn latent feature representation and causal learning to enforce the feature representation to form the direct causes and effects of the target variable that are invariant across domains. In addition to enable computer vision systems to be safely deployable in open-world environments, this project can significantly advance other fields of artificial intelligence (AI) and machine learning, including transfer learning, domain adaptation, and lifelong learning. The project will integrate research with education and involve undergraduate students in this research by leveraging the existing programs at Rensselaer Polytechnic Institute.This research explores a novel idea of integrating causality into deep neural networks to enhance generalization, fairness, and explanation. The project employs the Causal Markov Blanket (CMB) to capture the underlying causal mechanism in the data. It further demonstrates the optimal properties of the CMB features in predicting the target variable. Finally, it introduces an end-to-end deep learning framework to efficiently learn the CMB features for a target variable without causal sufficiency assumption. The project will evaluate the framework on some computer vision tasks. Despite its significant potential, this project investigates a relatively less explored area of study in computer vision and machine learning. The goal of the project is to generate preliminary results to demonstrate its feasibility and to identify areas of improvements.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

深度学习的最新发展导致了计算机粘膜和其他领域的显着进步。但是，当前的计算机视觉系统在与培训环境不同的新环境中的表现不佳。这个问题，如果没有解决，可能会导致预测错误，从而对生命危机和高风险应用（例如自动驾驶，机器人技术和制造）产生重大影响。为了应对这一挑战，该项目使用潜在功能介绍了一个新颖的深度学习框架。与大多数可观察到的功能不同，潜在特征是隐藏的。例如，在对象识别任务中，潜在因素可以是描述从图像的像素和每个训练图像的标签中提取的对象的规模和姿势。该框架利用深度学习来学习潜在的特征表示和因果学习来强制执行特征表示形式，以形成跨域的目标变量的直接原因和效果。除了使计算机视觉系统能够在开放世界环境中安全部署外，该项目还可以大大推动其他人工智能（AI）和机器学习领域，包括转移学习，域名适应和终身学习。该项目将通过利用Rensselaer理工学院的现有计划来将研究与教育联系起来，并让本科生参与这项研究。这项研究探讨了将热量纳入深层神经网络以增强概括，公平性和解释的新颖想法。该项目采用因果马尔可夫毛毯（CMB）来捕获数据中的基本催化机制。它进一步证明了CMB特征在预测目标变量时的最佳特性。最后，它引入了一个端到端的深度学习框架，以有效地学习目标变量的CMB功能而没有因果满意度假设。该项目将在某些计算机视觉任务上评估框架。尽管具有巨大的潜力，但该项目仍研究了计算机视觉和机器学习方面的研究领域相对较少的研究领域。该项目的目的是产生初步结果，以证明其可行性并确定改进领域。该奖项反映了NSF的法定任务，并通过使用基金会的知识分子优点和更广泛的影响审查标准来评估被认为是宝贵的支持。