RI: Small: Computational analysis of eye movements in reading: reader characteristics, cognitive state, and natural language processing

RI：小：阅读中眼动的计算分析：读者特征、认知状态和自然语言处理

• 批准号: 1815529
• 负责人: Roger Levy
• 金额: $ 40万
• 依托单位: Massachusetts Institute of Technology
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-08-01 至 2021-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1815529&HistoricalAwards=false
• 关键词: RI; Small; Computational; analysis; eye

Reading is the most widely practiced skill in the world. It occupies many hours of our daily lives and is crucial to functioning successfully in modern society. When we read, our eyes move over the text in a way that reflects how we perceive, process, and understand it. This project uses eye tracking to develop a new approach to studying how our eyes move during reading and what eye movement patterns can reveal about readers' linguistic knowledge and how they interact with text. In particular, the researchers develop eye tracking-based methods that automatically determine readers linguistic proficiency, how well they understand the text, and how difficult they find it. The project improves automatic text processing by machines, by taking into account information about how humans read. This research program benefits society by advancing our scientific understanding of language processing during reading and enabling new technologies that support human readers from a wide range of backgrounds and skill levels. It lays the foundations for future digital platforms that make it easier for people to access textual information, improve literacy, learn new languages, and personalize text content and complexity according to readers' needs and goals.The project introduces a novel conceptual and computational approach to studying human reading with both native and non-native speakers, by leveraging broad coverage eye movement patterns during reading of free-form text. It develops a computational framework that connects eye movement in reading to linguistic properties of the text, the readers' linguistic knowledge and their cognitive state during reading. To realize this framework, the project first focuses on using eye movement in reading to automatically predict readers' linguistic proficiency and estimate their comprehension of specific parts of a text. These and other related tasks help to characterize how language comprehension manifests in gaze and unfolds over time, and also lead to the development of a predictive computational framework for native and non-native reading. Its second focus is on integrating data and representations from eye tracking in natural language processing, with the aim of developing applications which support and enhance human reading and language learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

阅读是世界上最广泛练习的技能。它占据了我们日常生活的许多时间，对于现代社会的成功运作至关重要。当我们阅读时，我们的眼睛在文本上移动的方式反映了我们如何感知、处理和理解它。该项目使用眼动追踪开发一种新方法来研究我们的眼睛在阅读过程中如何移动，以及哪些眼动模式可以揭示读者的语言知识以及他们如何与文本互动。特别是，研究人员开发了基于眼动追踪的方法，可以自动确定读者的语言熟练程度、他们对文本的理解程度以及他们发现文本的难度。该项目通过考虑人类如何阅读的信息，改进了机器的自动文本处理。该研究项目通过推进我们对阅读过程中语言处理的科学理解，并启用支持不同背景和技能水平的人类读者的新技术，造福社会。它为未来的数字平台奠定了基础，使人们更容易访问文本信息，提高读写能力，学习新语言，并根据读者的需求和目标个性化文本内容和复杂性。该项目引入了一种新颖的概念和计算方法通过在阅读自由格式文本时利用广泛的眼动模式，研究母语者和非母语者的人类阅读。它开发了一个计算框架，将阅读中的眼球运动与文本的语言特性、读者的语言知识及其在阅读过程中的认知状态联系起来。为了实现这个框架，该项目首先侧重于在阅读中使用眼动来自动预测读者的语言熟练程度并估计他们对文本特定部分的理解。这些和其他相关任务有助于表征语言理解如何在凝视中体现并随着时间的推移而展开，并且还导致了针对母语和非母语阅读的预测计算框架的开发。它的第二个重点是在自然语言处理中整合来自眼动追踪的数据和表示，目的是开发支持和增强人类阅读和语言学习的应用程序。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准。

项目成果

Predicting Text Readability from Scrolling Interactions

通过滚动交互预测文本可读性

• 发表时间: 2021
• 期刊: Proceedings of the Conference on Natural Language Learning
• 作者: Gooding, Sian;Berzak, Yevgeni;Mak, Tony;Sharifi, Matt
• 通讯作者: Sharifi, Matt

Eye Movement Traces of Linguistic Knowledge

语言知识的眼动痕迹

• 发表时间: 2021
• 期刊: Proceedings of the Annual Conference of the Cognitive Science Society
• 作者: Berzak, Yevgeni;Levy, Roger P.
• 通讯作者: Levy, Roger P.

STARC: Structured Annotations for Reading Comprehension

• DOI: 10.18653/v1/2020.acl-main.507
• 发表时间: 2020-04
• 作者: Yevgeni Berzak;J. Malmaud;R. Levy

Bridging Information-Seeking Human Gaze and Machine Reading Comprehension

连接寻求信息的人类凝视和机器阅读理解

• 发表时间: 2020
• 期刊: Proceedings of the Conference on Natural Language Learning
• 作者: Malmaud, Jonathan;Levy, Roger P.;Berzak, Yevgeni
• 通讯作者: Berzak, Yevgeni