Data-Driven Appearance Transfer for Realistic Image Synthesis

用于真实图像合成的数据驱动的外观传输

• 批准号: 0541230
• 负责人: Alexei Efros
• 金额: --
• 依托单位: Carnegie-Mellon University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2006
• 资助国家: 美国
• 起止时间: 2006-02-01 至 2009-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0541230&HistoricalAwards=false
• 关键词: Data; Driven; Appearance; Transfer; Realistic

Realistic image synthesis is a central goal of computer graphics. Major recent advances have allowed researchers to model a wide spectrum of complicated visual phenomena with a very high degree of realism. Yet, even the best computer-generated feature films are a far cry from what one might consider "real". Curiously, the problem is generally not with computer graphics being unable to model the physics of the everyday visual world -- the problem is with the world itself! It's just too complex, too noisy, too rich and vivid to be recreated from scratch by even the most skilled and patient artist.One solution is to use image-based methods and directly capture visual appearance of everything in the world -- if only it was feasible. Instead, this research effort centers on transferring appearance from a large database of stored visual data into a novel scene. The reason is that while capturing details of a particular scene is very expensive and time-consuming, obtaining similar information from some relevant scene is relatively easy. There is a tremendous amount of visual data that is already captured and available - thousands of webcams all over the world, millions of photographs placed on the Internet, depicting anything from sandstorms in Sahara to the glaciers in Alaska. And more data is being added every day. Our research is developing a unified approach for appearance transfer. Two broad scenarios are considered: transfer in image stacks (e.g. webcams) and single image transfer. In both cases, the major research issues involve: (1) grouping images and image stacks into regions with coherent material/geometry properties, (2) determining correspondence between various groups in the scene and the database, (3) and finally transferring the correct appearance from the database by combining it with the large-scale structure of the input scene.

逼真的图像合成是计算机图形的中心目标。最近的重大进展使研究人员能够以非常高的现实主义对广泛的复杂视觉现象进行建模。 但是，即使是最好的计算机生成的故事片也与可能“真实”的哭声相去甚远。 奇怪的是，问题通常不是因为计算机图形无法对日常视觉世界的物理进行建模 - 问题与世界本身有关！它太复杂了，太嘈杂，太丰富又生动，甚至无法由最熟练和耐心的艺术家从头开始重新创建。一种解决方案是使用基于图像的方法并直接捕获世界上所有事物的视觉外观 - 如果它是可行的。取而代之的是，这项研究工作集中在将存储的视觉数据的大数据库转移到新型场景中。 原因是，尽管捕获特定场景的详细信息非常昂贵且耗时，因此从某些相关场景中获取相似的信息相对容易。 已经捕获和可用的是大量的视觉数据 - 世界各地成千上万的网络摄像头，在互联网上放置了数百万张照片，描绘了从撒哈拉沙漠的Sandstorms到阿拉斯加的冰川。 每天都在添加更多数据。我们的研究正在开发一种统一的外观转移方法。 考虑了两个广泛的方案：图像堆栈中的传输（例如网络摄像头）和单图像传输。 在这两种情况下，主要的研究问题都涉及：（1）将图像和图像堆栈分组为具有连贯的材料/几何特性的区域，（2）确定场景中各组与数据库之间的对应关系，（3），最后通过将其与输入场景的大规模结构相结合，从数据库中转移正确的外观。