CAREER: Understanding visual reasoning for visual communication

职业：理解视觉传达的视觉推理

• 批准号: 1945303
• 负责人: Karen Schloss
• 金额: $ 55.87万
• 依托单位: University of Wisconsin-Madison
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-07-01 至 2025-06-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1945303&HistoricalAwards=false
• 关键词: CAREER; Understanding; visual; reasoning; communication

This project aims to advance the understanding of visual reasoning for visual communication. Visual reasoning enables people to translate visual input to abstract concepts. For example, to interpret which counties will receive more snowfall using a weather map, it is necessary to figure out which colors on the weather map indicate which amounts of snowfall. People have expectations about how visual features should map to concepts in visualizations, and it is harder for them to interpret visualizations that violate those expectations, even if mappings are clearly labeled. However, the nature of those expectations and their role in visual reasoning is not well-understood, so the design of information visualizations is often unprincipled and ad-hoc. With a better understanding of how visual reasoning works, it will be possible to design visualizations that fit its strengths and optimize visual communication. The investigators will address this problem by studying how people infer meaning from color in visualizations. This research can be translated to producing online tools for designing visualizations, which will improve STEM education and increase public literacy and engagement with science and technology. The education plan will use visual communication to make science more accessible and engaging through virtual reality (VR) and accompanying hands-on experiences with color and visualization, for both college undergraduates and middle-school students. The investigators will also support broadening participation of females in STEM through mentoring among the PI, graduate student, undergraduate intern, and middle school girls. The proposed research will address fundamental questions about how visual reasoning enables visual communication. In Objective 1, the investigators will study how people learn to associate perceptual features with novel concepts through environmental statistics. They will also examine how learned associations extend beyond perceptual input due to categorical structure in cognitive representations. In Objective 2, the team will study how to automatically estimate color–concept associations from image statistics. They will construct and evaluate new models that incorporate predictors based on visual input and cognitive representations of color categories. In the process of constructing these models, the team will develop new methods for quantifying graded category membership in a continuous space. By modeling human judgments, there is potential to develop new insights into how those judgments are made. In Objective 3, the team will investigate how people use color–concept associations to interpret visualizations, in a process called assignment inference. Assignment inference enables people to infer optimal mappings between colors and concepts in visualizations, but little is known about how this process works. It is proposed that it can be understood using accumulator models typically used in decision making research, which will forge new connections between the fields of decision making and information visualization. The results will increase knowledge of how people integrate information from multiple, conflicting sources and provide new knowledge about how to optimally design semantically interpretable visualizations.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目旨在提高对视觉推理的视觉交流的理解。视觉推理使人们能够将视觉输入转化为抽象概念。例如，要使用天气图解释哪些县将获得更多的降雪，有必要确定天气图上的哪些颜色指示哪些降雪量。人们对视觉特征应如何映射到可视化的概念有期望，即使映射清楚地标记了映射，他们也很难解释违反这些期望的可视化效果。但是，这些期望的性质及其在视觉推理中的作用并不理解，因此信息可视化的设计通常是无关紧要的和临时的。有了更好地了解视觉推理的工作原理，可以设计适合其优势并优化视觉交流的可视化。研究人员将通过研究人们如何从可视化中推断出含义来解决这个问题。这项研究可以转化为生产用于设计可视化的在线工具，这将改善STEM教育并提高公众识字和对科学技术的参与。该教育计划将使用视觉沟通来使科学通过虚拟现实（VR）更容易获得和参与，并为大学的本科生和中学生而言，参与色彩和可视化的动手体验。调查人员还将通过PI，研究生，本科实习生和中学女生之间的心理来支持扩大女性在STEM中的参与。拟议的研究将解决有关视觉推理如何实现视觉交流的基本问题。在目标1中，研究人员将研究人们如何通过环境统计数据将感知特征与新颖概念联系起来。他们还将研究学到的关联如何超越认知表示中的分类结构而扩展。在目标2中，团队将研究如何自动从图像统计数据估算颜色 - 概念关联。他们将根据颜色类别的视觉输入和认知表示形式构建和评估新模型，这些模型结合了预测因素。在构建这些模型的过程中，团队将开发新的方法，以量化连续空间中的分级类别成员资格。通过对人类法官进行建模，有可能对这些法官的制作方式发展新的见解。在目标3中，团队将在一个称为分配推理的过程中调查人们如何使用颜色概念关联来解释可视化。分配推论使人们能够在可视化中推断颜色和概念之间的最佳映射，但是对于此过程的工作方式知之甚少。建议使用通常在决策研究中使用的累加器模型来理解它，这将在决策制定和信息可视化领域之间建立新的联系。结果将增加有关人们如何从多个，冲突来源整合信息的知识，并提供有关如何最佳设计语义上可解释的可视化的新知识。该奖项反映了NSF的法定任务，并通过使用基金会的智力优点和更广泛的影响来评估NSF的法定任务。

项目成果

A Holey Perspective on Venn Diagrams

维恩图的漏洞视角

• DOI: 10.1111/cogs.13073
• 发表时间: 2021
• 期刊: Cognitive Science
• 影响因子: 2.5
• 作者: Bartel, Anna N.;Lande, Kevin J.;Roos, Joris;Schloss, Karen B.
• 通讯作者: Schloss, Karen B.

Context Matters: A Theory of Semantic Discriminability for Perceptual Encoding Systems

• DOI: 10.1109/tvcg.2021.3114780
• 发表时间: 2021-08
• 期刊: IEEE Transactions on Visualization and Computer Graphics
• 影响因子: 5.2
• 作者: Kushin Mukherjee;Brian Yin;Brianne E. Sherman;Laurent Lessard;Karen B. Schloss

Colour-concept association formation for novel concepts

• DOI: 10.1080/13506285.2022.2089418
• 发表时间: 2022-07-21
• 期刊: VISUAL COGNITION
• 影响因子: 2
• 作者: Schoenlein，Melissa A.;Schloss，Karen B.
• 通讯作者: Schloss，Karen B.