New computational models of human visual perception of surface colour, 3D shape, and lighting

人类视觉感知表面颜色、3D 形状和照明的新计算模型

• 批准号: RGPIN-2022-04583
• 负责人: Murray, Richard
• 金额: $ 2.04万
• 依托单位: York University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=766062
• 关键词: New; computational; models; human; visual

What could be easier than seeing? We see without trying, and usually without thinking about it. Seeing seems easy because our visual cortex is a powerful computing device with a lifetime of experience. One reason why vision is a challenging computational task, though, is that all images are highly ambiguous: any given image could conceivably be seen as depicting a wide range of shapes, colours, and lighting conditions. In order to perceive things correctly, our visual system must overcome this ambiguity. The long-term goal of my research program is to develop computational models that solve this problem in the same way people do, and see what people see in complex, realistic scenes. The research proposed here will approach this goal in two ways. First, using powerful new methods developed for machine learning and computer vision, I will train artificial neural networks to perceive surface colour, 3D shape, and lighting conditions in complex scenes. To do this I will use rendering software to generate training data that includes images of many diverse scenes, along with representations that show the true colour and shape of objects in those scenes, as well as the true lighting conditions. The networks will be trained to view just the images, and deduce the shape, colour, and lighting in the scenes as well as possible. Previous research suggests that simply because the networks are trained on naturalistic data, they will have some of the same characteristics as human vision: they will find the same visual tasks easy or hard, and they will see many of the same illusions. I will test this prediction, and as it will probably not turn out to be completely true, I will also revise the networks as necessary so that they see shape, colour, and lighting as similarly to human vision as possible. The second approach I will take is to run perceptual experiments with human participants that investigate what fundamental visual features make up our visual world. For example, we obviously perceive colour and 3D shape, and just as obviously we do not perceive the polarization of light. I will focus these experiments on the claim that 'brightness' is a fundamental perceptual dimension, defined as the point-by-point intensity of images (technically, 'perceived luminance'). I will systematically vary surface colour and lighting of test patches in real and computer-generated scenes, and measure how these variations affect judgements of surface colour and brightness. These measurements will help to establish whether 'brightness' is a feature that we actually perceive, separate from surface colour and lighting conditions. This research will help us to understand normal human vision, both in real life and in the simulated virtual environments that are becoming increasingly important for many applications. It will also provide information that will be useful for developing computer vision systems that see what people see in complex, realistic scenes.

有什么比看到容易的？我们看到不尝试，通常不考虑它。看到看起来很容易，因为我们的Visual Cortex是具有一生经验的功能强大的计算设备。视觉是一项具有挑战性的计算任务的原因之一是，所有图像都高度模棱两可：可以想象，任何给定的图像都可以看作描绘了各种形状，颜色和照明条件。为了正确看待事物，我们的视觉系统必须克服这种歧义。我的研究计划的长期目标是开发以人们相同的方式解决此问题的计算模型，并在复杂，现实的场景中看到人们看到的内容。这里提出的研究将以两种方式实现这一目标。首先，使用用于机器学习和计算机视觉的强大新方法，我将在复杂场景中训练人造神经网络以感知表面颜色，3D形状和照明条件。为此，我将使用渲染软件来生成培训数据，其中包括许多不同场景的图像，以及在这些场景中显示对象的真实色彩和形状以及真实的照明条件的表示。将训练网络以观看图像，并尽可能在场景中推断出形状，颜色和照明。先前的研究表明，仅仅因为网络经过自然主义数据的培训，它们将具有与人类视野相同的特征：他们会轻松或艰难地发现相同的视觉任务，并且会看到许多相同的幻觉。我将测试这一预测，并且可能不会完全正确，因此我还将根据需要修改网络，以便它们看到形状，颜色和照明与人类视野尽可能相似。我将采用的第二种方法是与人类参与者进行感知实验，以研究哪些基本视觉特征构成了我们的视觉世界。例如，我们显然会感知颜色和3D形状，并且显然我们不感知光的极化。我将把这些实验集中在说“亮度”是一个基本知觉维度的说法上，定义为图像的逐点强度（从技术上讲，“可感知的亮度”）。我将在真实和计算机生成的场景中系统地改变表面颜色和测试贴片的照明，并测量这些变化如何影响表面颜色和亮度的判断。这些测量结果将有助于确定“亮度”是否是我们实际感知的特征，与表面颜色和照明条件分开。这项研究将有助于我们在现实生活中和模拟的虚拟环境中了解正常的人类视野，这些虚拟环境对许多应用变得越来越重要。它还将提供信息，这些信息对于开发计算机视觉系统有用，这些系统在复杂，现实的场景中看到了什么。