RI: Small: Understanding and Synthesizing People in 3D Scenes

RI：小：理解和合成 3D 场景中的人物

基本信息

批准号：
2008313
负责人：
Noah Snavely
金额：
$ 49.86万
依托单位：
Cornell University
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2020
资助国家：
美国
起止时间：
2020-10-01 至 2024-09-30
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2008313&HistoricalAwards=false
关键词：
RI Small Understanding Synthesizing People

项目摘要

This project advances fundamental research into understanding people in images, and how they interact with their environment. This goal is important because people are crucial in many real-world applications that involve imagery. For example, vehicles and robots should perceive people, predict their behavior, and operate safely around them. In the realm of computer graphics, algorithms that generate content for movies and games should depict people with realistic appearance and who interact with their environment in realistic ways. Developing a computational understanding of people sufficient for such applications will involve, crucially, understanding the connection between people and the scenes they inhabit. For instance, in a typical street scene, people will tend to appear walking on sidewalks or crosswalks -- but if it suddenly starts raining, they might look and act differently, for instance, by carrying umbrellas, hurrying for shelter, etc. This project seeks to build such a computational understanding of people in scenes via new human-centric methods for perceiving the world. The research will also be coupled with educational activities, including efforts to broaden participation in computing.The technical goals of the project fall into two main thrusts corresponding to a computer vision and a computer graphics goal: (1) human-centric understanding of images, and (2) realistic synthesis of people into images. Both of these goals will be driven by new machine learning methods that will answer the following series of questions: given an image of a scene, where might a person appear, what would they be doing, and how would they look? These questions are distinct from the typical computer vision task of identifying where humans actually are in an image, and instead involve reasoning about where they could be. This reasoning is highly dependent on the scene depicted in the image -- are there benches, is it raining, etc. -- and also on the 3D geometry of the scene -- e.g., whether a particular surface is horizontal or vertical. Hence, the project will explore scene- and 3D-aware learning of people and their interactions with the world.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目推进了理解图像中的人以及他们如何与环境互动的基础研究。这个目标很重要，因为人在许多涉及图像的现实应用中至关重要。例如，车辆和机器人应该感知人类，预测他们的行为，并在他们周围安全运行。在计算机图形学领域，为电影和游戏生成内容的算法应该描绘具有真实外观的人物，并以真实的方式与环境互动。至关重要的是，要对人类进行足够的计算理解，以适应此类应用，就需要理解人类与其所居住的场景之间的联系。例如，在典型的街道场景中，人们往往会出现在人行道或人行横道上行走——但如果突然下雨，他们的表情和行为可能会有所不同，例如携带雨伞、匆忙寻找避难所等。这个项目寻求通过新的以人为中心的感知世界的方法来建立对场景中的人的计算理解。该研究还将与教育活动相结合，包括努力扩大对计算的参与。该项目的技术目标分为与计算机视觉和计算机图形目标相对应的两个主要目标：（1）以人为中心的图像理解， (2)将人物逼真地合成为图像。这两个目标都将由新的机器学习方法驱动，这些方法将回答以下一系列问题：给定一个场景图像，一个人可能出现在哪里，他们会做什么，以及他们看起来如何？这些问题与识别人类在图像中实际位置的典型计算机视觉任务不同，而是涉及推理他们可能在哪里。这种推理高度依赖于图像中描绘的场景（是否有长凳、是否下雨等）以及场景的 3D 几何形状（例如，特定表面是水平还是垂直）。因此，该项目将探索人们的场景和 3D 感知学习及其与世界的互动。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。