CAREER: Large-scale Appearance Modeling

职业：大型外观造型

• 批准号: 1350323
• 负责人: Pieter Peers
• 金额: $ 47.35万
• 依托单位: College of William and Mary
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2014
• 资助国家: 美国
• 起止时间: 2014-01-15 至 2019-12-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1350323&HistoricalAwards=false
• 关键词: CAREER; Large; scale; Appearance; Modeling

The visual appearance of the world around us is the result of complex light interactions between different surfaces and material properties that comprise a scene. Despite staggering advances in data-driven appearance modeling, the creation of accurate models of large environments remains an open problem. The reliance of most current appearance modeling methods on active lighting to probe different slices of a scene's appearance precludes their use in environments where there is limited or no control over the incident ambient lighting. Furthermore, to facilitate calibration, many appearance modeling techniques estimate the appearance of a scene from a fixed vantage point, excluding scenes too large to fit in a single view with sufficient detail. In this research, the PI will investigate two novel appearance modeling paradigms designed expressly for large-scale environments under uncontrolled ambient lighting: appearance-from-motion and appearance-by-similarity. The former exploits relations between observations from different viewpoints to infer the full reflectance behavior, while the latter seeks to identify the best match from a library of pre-existing appearance instances to a possibly under-constrained set of observations. To support these two paradigms, a novel appearance model will be developed that builds upon our intuitions regarding scene appearance. The work will focus on two common types of input: community photo-collections and targeted video sequences.Broader Impacts: This research will pave the way towards practical techniques for in-situ appearance modeling of large-scale environments, while stimulating new research in computer vision and in data-driven appearance modeling in computer graphics by answering fundamental questions as to whether we can model appearance from motion and/or by exploiting similarity. The project will have far-reaching impact not only on computer science but also on diverse fields ranging from metropolitan planning to cultural heritage to entertainment. The ability to model existing environments will be beneficial to various security and safety training programs (for example, virtual fire drill simulations of existing buildings and sites could help train and prepare firefighters and first responders). The emerging field of virtual reality therapy will also benefit from this research, by making it easier to create digital models of large-scale environments (so that, for example, patients who have suffered a stroke can practice motor rehabilitation skills in virtual reproductions of environments they encounter in their daily lives, while autistic children can train to improve their social interactions in virtual reproductions of places such as classrooms which they encounter in their daily lives).

我们周围世界的视觉外观是不同表面和构成场景的材料属性之间复杂的光相互作用的结果。 尽管数据驱动的外观建模方面取得了惊人的进步，但大型环境的准确模型的创建仍然是一个开放的问题。 大多数当前的外观建模方法在主动照明上探测场景外观的不同切片的依赖性排除了它们在对事件环境照明的有限或无法控制的环境中的使用。 此外，为了促进校准，许多外观建模技术从固定的有利位置估算了场景的外观，不包括太大的场景，无法单一的视图，并有足够的细节。 在这项研究中，PI将研究两个新颖的外观建模范式，在不受控制的环境照明下为大规模环境设计明确设计：外观 - 动作和相似性。 前者从不同的角度利用观察结果之间的关系来推断完整的反射率行为，而后者则试图从现有的外观实例的库中确定最佳匹配，并将其匹配到可能受到约束的一组观测值。 为了支持这两个范式，将开发出一种新颖的外观模型，该模型建立在我们有关场景外观的直觉上。 这项工作将重点关注两种常见的输入类型：社区照片集成和有针对性的视频序列。Broader的影响：这项研究将为大规模环境的现场外观建模迈出实用技术的方式，同时刺激计算机视觉中的新研究以及在数据驱动的外观模型中在计算机图形中通过计算机图形来探讨我们是否可以通过探索运动和/可以模拟运动的基本问题，或 该项目将不仅对计算机科学产生深远的影响，而且对从大都市规划到文化遗产到娱乐的各种领域也将对各种领域产生影响。 建模现有环境的能力将有助于各种安全和安全培训计划（例如，对现有建筑物和站点的虚拟消防钻模拟可以帮助培训和准备消防员和急救人员）。 虚拟现实疗法的新兴领域也将从这项研究中受益，使创建大规模环境的数字模型（例如，例如，遭受中风的患者可以在他们日常生活中遇到的虚拟环境中练习运动康复技能，而在日常生活中遇到的环境复制，而自闭症儿童可以在自闭症中培训他们的社交互动，以改善他们在小时候的社交互动中，而他们在日常生活中的社交互动，这是他们在经常生活中的经历。

项目成果

Interactive Curation of Datasets for Training and Refining Generative Models

• DOI: 10.1111/cgf.13844
• 发表时间: 2019-10
• 期刊: Computer Graphics Forum
• 影响因子: 2.5
• 作者: Wenjie Ye;Yue Dong;P. Peers

Deep inverse rendering for high-resolution SVBRDF estimation from an arbitrary number of images

• DOI: 10.1145/3306346.3323042
• 发表时间: 2019-07
• 期刊: ACM Transactions on Graphics (TOG)
• 作者: Duan Gao;Xiao Li;Yue Dong;P. Peers;Kun Xu;Xin Tong

Synthesizing 3D Shapes From Silhouette Image Collections Using Multi-Projection Generative Adversarial Networks

• DOI: 10.1109/cvpr.2019.00568
• 发表时间: 2019-06
• 期刊: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
• 作者: Xiao Li;Yue Dong;P. Peers;Xin Tong