CAREER: Multimodal Language Processing for Natural Interfaces

职业：自然界面的多模态语言处理

• 批准号: 9876223
• 负责人: Michael Johnston
• 金额: $ 33.39万
• 依托单位: Oregon Health & Science University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 1999
• 资助国家: 美国
• 起止时间: 1999-03-01 至 2005-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=9876223&HistoricalAwards=false
• 关键词: CAREER; Multimodal; Language; Processing; Natural

By providing a number of channels through which information may pass between user and computer, multimodal interfaces promise to significantly increase the bandwidth and fluidity of the interface between humans and machines. Natural language processing techniques typically focus on processing strings of words within a single dimension. This program of research seeks to develop an approach to multimodal language processing which enables parsing and interpretation of natural human input distributed across two or three spatial dimensions, time, and the acoustic dimension of speech. A declarative unification-based grammar representation is proposed for the description of multimodal integration strategies. This representation is interpreted by a multidimensional parser yielding an architecture which provides the foundation for next generation natural interfaces supporting human-computer interaction through speech, pen, gesture, and other modes. Fundamental advances in multimodal interfaces are essential to the successful migration of human-computer interaction from the desktop to small mobile computing devices, public information kiosks, and other platforms for which complex graphical user interfaces are not feasible. Multimodal interfaces stand to play a critical role both in realizing the vision of universal access to information and in the development of assistive technologies for people with disabilities.

通过提供许多信息可以在用户和计算机之间传递的通道，多模式接口有望显着增加人与机器之间接口的带宽和流动性。 自然语言处理技术通常专注于处理单个维度内的字符串。 该研究项目旨在开发一种多模态语言处理方法，能够解析和解释分布在两个或三个空间维度、时间和语音声学维度上的自然人类输入。 提出了一种基于声明性统一的语法表示来描述多模态集成策略。 这种表示由多维解析器解释，产生一种架构，该架构为下一代自然界面提供了基础，支持通过语音、笔、手势和其他模式进行人机交互。 多模式界面的根本性进步对于将人机交互从桌面成功迁移到小型移动计算设备、公共信息亭和其他复杂图形用户界面不可行的平台至关重要。 多模式界面在实现普遍获取信息的愿景和开发残疾人辅助技术方面都将发挥关键作用。