CHS: Small: Collaborative Research: Articulate+ - A Conversational Interface for Democr atizing Visual Analysis

CHS：小型：协作研究：Articulate - 用于民主化视觉分析的对话界面

• 批准号: 2007257
• 负责人: Andrew Johnson
• 金额: $ 33.3万
• 依托单位: University of Illinois at Chicago
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-10-01 至 2023-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2007257&HistoricalAwards=false
• 关键词: CHS; Small; Collaborative; Research; Articulate+

The Internet gives the public, including scientists, access to vast amounts of data. Visualization is a powerful tool to take that data and turn it into a form that is easier to understand, helping people make better decisions. However, creating good visualizations from data is not easy. The Articulate+ Project will work to generate new knowledge about how to empower people with information visualization, by using speech and gesture as inputs, in new ways. A goal is to enable users to benefit from imprecise statements about what they want, rather than needing to give specific commands telling the computer what to do. This project will make it easier for citizens to engage in conversations with the computer about economic, medical data, demographic, transportation data, climate, and sports data; to answers that are more meaningful, and so to democratize data literacy. This research will use speech, gesture, and log data as new input interaction modalities for visual analytics. It will develop new understandings of how the imprecise and vague nature of natural language queries and gestures, as contextualized by work on a specific visual analysis problem, can be modeled as an opportunity for an intelligent software system to learn more about the underlying intent of the users. This model of intent, in turn, will be used to provide contextualized visualizations, which are expected to help those users gain valuable insights from their data. The audio speech data will be used to computationally model overhearing, to help infer users' current contextualized goals. The gesture data will be used to disambiguate expressions that refer to visual elements of a visualization (e.g., "that pie chart about electricity consumption"). The project will develop new understandings of how to combine the log of visualization state, the computational model of overhearing, and the gesture data in order to generate new visualizations in support of users’ work on tasks. Evaluations will be performed in both controlled laboratory and naturalistic study settings to determine whether effective semantic parsers can be derived for these specific visual analysis domains, and how this contextual interface affects users’ experience of visualization and discovery. The integrated annotated datasets from the studies will be made available to the research community satisfying a need for ecologically valid situated language interaction corpora.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

互联网为包括科学家在内的公众提供了获取大量数据的强大工具，可以将这些数据转化为更容易理解的形式，从而帮助人们做出更好的决策。 Articulate+ 项目将致力于通过使用语音和手势作为输入，以新的方式生成关于如何赋予人们信息可视化的新知识，目标是使用户能够从关于他们想要的内容的不精确陈述中受益。而不是需要给出特定的命令该项目将使公民更容易与计算机进行有关经济、医疗数据、人口、交通数据、气候和体育数据的对话，从而获得更有意义的答案；这项研究将使用语音、手势和日志数据作为视觉分析的新输入交互方式，它将对自然语言查询和手势的不精确和模糊性质如何通过特定视觉工作的语境化产生新的理解。分析问题，可以建模为智能软件系统有机会更多地了解用户的潜在意图，而这种意图模型将用于提供情境化的可视化，这有望帮助这些用户从他们的音频语音中获得有价值的见解。数据将用于计算无意听到的模型，以帮助推断用户当前的情境目标。手势数据将用于消除涉及可视化视觉元素的表达的歧义（例如，“有关电力消耗的饼图”）。将开发新的了解如何将可视化状态日志、无意听到的计算模型和手势数据结合起来，以生成新的可视化以支持用户的任务工作。评估将在受控实验室和自然研究环境中进行，以确定。是否可以为这些特定的视觉分析领域导出有效的语义解析器，以及该上下文界面如何影响用户的可视化和发现体验。研究中的集成注释数据集将提供给研究界，以满足生态有效定位的需求。语言互动该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

An Investigation into an Always Listening Interface to Support Data Exploration

• DOI: 10.1145/3581641.3584079
• 发表时间: 2023-03
• 期刊: Proceedings of the 28th International Conference on Intelligent User Interfaces
• 作者: Roderick S. Tabalba;Nurit Kirshenbaum;J. Leigh;Abari Bhattacharya;Veronica Grosso;Barbara Di Eugenio;Andrew E. Johnson;Moira L. Zellner

Reference Resolution and New Entities in Exploratory Data Visualization: From Controlled to Unconstrained Interactions with a Conversational Assistant

• DOI: 10.18653/v1/2023.sigdial-1.33
• 发表时间: 2023
• 作者: Abari Bhattacharya;Abhinav Kumar;Barbara Di Eugenio;Roderick S. Tabalba;Jillian Aurisano;Veronica Grosso;Andrew Johnson;Jason Leigh;Moira L. Zellner

Ditto: Exploring data in large display environments through speech+mid-air gesture interactions

同上：通过语音空中手势交互探索大型显示环境中的数据

• 发表时间: 2022
• 期刊: IEEE Vis 2022
• 作者: Aurisano, J.;Alsaiari, A.;Kumar, A.;Zellner, M;Leigh, J.;Di Eugenio, B.;Johnson, A.
• 通讯作者: Johnson, A.