CAREER: Art and Vision: Scene Layout from Pictorial Cues

职业：艺术与视觉：根据图片提示进行场景布局

• 批准号: 1257700
• 负责人: Stella Yu
• 金额: $ 15.1万
• 依托单位: International Computer Science Institute
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2012
• 资助国家: 美国
• 起止时间: 2012-04-01 至 2013-12-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1257700&HistoricalAwards=false
• 关键词: CAREER; Art; Vision; Scene; Layout

CAREER: Art and Vision: Scene Layout from Pictorial CuesPI: Stella (XingXing) Yu Institution: Boston CollegeArtists are the masters of visual perception. Studying art and vision together can provide new solutions to fundamental problems in computer vision. We focus on inferring scene layout from a single image. This problem has been studied since the earliest days of Artificial Intelligence research, resulting in a host of so-called Shape-from-X methods, where X could be shading, perspective, etc.Unfortunately, each of these methods works under its own assumptions which often do not hold in real images. How these cues interact and integrate remains elusive. Painters constantly use a combination of four techniques: occlusion, perspective, shading, and form to effectively evoke a 3D percept from a 2D picture. Studying their techniques can lend insights into the computation of recovering scene layout from pixel values. The PI proposes to bring artists and vision scientists together to solve the computational problem of scene layout from pictorial cues. This project realizes it in three areas:education, experiments and computational modeling.A new interdisciplinary course, Art and Visual Perception, has been developed at Boston College to give a comprehensive cross-examination of how art contributes to the understanding of vision, and how vision contributes to the generation and viewing of art. Students are actively engaged in both art practice and vision experiments.Learning art and vision together results in a deeper understanding than studying each discipline separately. Students' assignments also result in valuable datasets for vision research.The computational approach to scene layout from pictorial cues in this project is to group pixels into spatially organized surfaces from a global integration of multiple pictorial cues in a spectral graph-theoretic framework. The goal is to turn artistic rendering knowledge on how these cues interact into a computational reality.The PI will study geometry (occlusion and perspective), appearance (brightness and color), and form using eye tracking and psychophysics experiments and computational models. These efforts are organized into two phases that progress from inferring the spatial layout from scenes made of planar surfaces (rooms and streets) to scenes made of curved surfaces (landscape and generic scenes).Intellectual MeritWhat is most remarkable about vision is its ability to perceive 3D spatial layout from a single 2D image. The proposed research replicates this ability in computation from a grouping perspective.Compared to statistical learning approaches, the grouping method is not only generic and thus scales well with the number of scenes, but can also produce a precise organization of surfaces in the scene.Compared to traditional Shape-from-X approaches, the grouping method examines each pictorial cue in conjunction with others. The integration of these multiple pictorial cues allows them for the first time to become applicable to real images. The PI has developed the essential grouping machinery in spectral graph theory for depth segregation. Compared to most existing formulations on this topic, it has unparalleled conceptual simplicity, computational efficiency, and guaranteed near-global optimality. The proposed research on brightness and color perception, in connection with Shape-from- Shading and surface organization, will help clarify the role of low- level and high-level mechanisms in the long-standing scientific debate between Hering and Helmholtz on color perception.Broader ImpactThis project bridges the gap between art and science not only in research but also in education by developing a new curriculum that traverses the areas of neuroscience, psychology, computer science, and visual arts, by involving students in art practice and scientific experiments, and by providing a forum for artists and scientists to exchange ideas on visual perception. These interdisciplinary efforts befit the liberal arts education tradition at Boston College. This project will not only benefit from the strong Fine Arts department on campus, but also cultivate computer science awareness and outreach to non-technical people, and promote the growth of the young Computer Science department at Boston College.URL: http://www.cs.bc.edu/~syu/artvis/

职业：艺术与视觉：图片cuespi的场景布局：Stella（Xingxing）Yu Institution：波士顿大学主义者是视觉感知的大师。一起研究艺术和视野可以为计算机视觉中的基本问题提供新的解决方案。我们专注于从单个图像推断场景布局。自人工智能研究的最早几天以来，就已经研究了这个问题，从而导致了许多所谓的形状 - X方法，其中X可能是阴影，透视等。这些线索如何相互作用和集成仍然难以捉摸。画家不断使用四种技术的组合：遮挡，透视，阴影和形式，从2D图片中有效地唤起3D感知。研究他们的技术可以为从像素值恢复场景布局的计算提供洞察力。 PI建议将艺术家和视觉科学家聚集在一起，以解决图片提示中场景布局的计算问题。该项目在三个领域中实现了这一点：教育，实验和计算建模。在波士顿学院开发了新的跨学科课程，艺术和视觉感知，以全面盘问艺术如何促进对视觉的理解以及视觉如何为艺术的产生和观看。学生们积极参与艺术实践和视觉实验。学习艺术和愿景会导致比单独研究每个学科更深入地了解。学生的作业还为视觉研究提供了宝贵的数据集。该项目中的图形提示的场景布局的计算方法是将像素从光谱图理论框架中的多个绘画线索的全球整合中分组为空间组织的表面。目的是将有关这些线索如何相互作用的艺术渲染知识转变为计算现实。PI将研究几何（遮挡和透视图），外观（亮度和色彩），并使用眼科跟踪和心理物理学实验和计算模型形成。这些努力分为两个阶段，从从平面表面（房间和街道）的场景中推断出空间布局到由曲面（景观和通用场景）制成的场景。智能优点最引人注目的视觉是其从单个2D图像中感知3D空间布局的能力。拟议的研究从分组的角度从统计学习方法中复制了计算能力，分组方法不仅是通用的，因此与场景的数量相比可以很好地扩展，而且还可以在场景中生成一个精确的表面组织，以与传统的形状 - X方法相比，分组方法研究每种图形提示，以与其他串联相结合。这些多个图形提示的集成使它们首次适用于真实图像。 PI在光谱图理论中开发了基本的分组机制，以进行深度分离。与该主题上的大多数现有配方相比，它具有无与伦比的概念简单性，计算效率和可保证的近乎全球最优性。关于亮度和色彩感知的拟议研究，与形状与遮盖和表面组织有关，将有助于阐明低水平和高级机制在赫林和赫尔姆霍尔兹之间的长期科学辩论中的作用，在色彩感知上对颜色感知之间的科学辩论。科学和视觉艺术，通过让学生参与艺术实践和科学实验，并为艺术家和科学家提供一个论坛，以交流有关视觉感知的想法。这些跨学科的努力符合波士顿学院的文科教育传统。该项目不仅将从校园的强大美术系中受益，而且还将培养计算机科学意识和向非技术人员培养，并在波士顿学院促进年轻计算机科学系的成长：