Multimodal natural assistive interfaces to support reading, learning, and access to critical information

多模式自然辅助界面，支持阅读、学习和访问关键信息

• 批准号: RGPIN-2016-06308
• 负责人: Munteanu, Cosmin
• 金额: $ 1.6万
• 依托单位: University of Toronto
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2019
• 资助国家: 加拿大
• 起止时间: 2019-01-01 至 2020-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=678803
• 关键词: Multimodal; natural; assistive; interfaces; support; Multimodal; natural; assistive; interfaces; support

“Traditional” user interfaces are continuously being replaced by mobile, touch, and speech-based interaction paradigms. These have helped us and empowered us; yet mobile interfaces have also marginalized many vulnerable user groups. For some, such as low-literacy adults or users with limited eyesight, even simple tasks (e.g. reading a book, understanding a user manual, comprehending a health pamphlet) can be a struggle despite the potential for assistance afforded by smart, portable, and personal interfaces.******The goal of my research is to provide practical support to marginalized users struggling to perform essential activities such as reading and acquiring critical information. For this, I propose to investigate and develop intelligent assistive technologies based on mobile, collaborative, and multimodal interfaces (speech, natural language, and gestural).******During my previous research I have extensively studied the human factors of using imperfect Automatic Speech Recognition (ASR) and Natural Language Processing (NLP) systems, and have contributed to facilitating natural interactions (such as gestures) with information-rich media and computing devices. I have shown that such interfaces, if well adapted to the needs of their users, can provide much-needed assistance to marginalized users. In turn, the underlying technology that enables such multimodal systems can improve when its design and functionality is informed by users' needs and limitations. I plan to continue following the same approach of combining interaction design and ASR/NLP research.******The goal of this program is to research methods and designs for interactive speech, language, and gesture-based mobile assistive interfaces that provide practical support to users struggling to perform essential activities such as reading, learning, and acquiring critical information. I plan to investigate multimodal and collaborative, semi-automated, and interactive annotation and information extraction mechanisms that will assist users in acquiring essential knowledge from electronic documents. This research, encompassing a wide range of speech and language-enabled media, will address the needs of a diverse set of marginalized and underserved user groups, from low-literacy adults trying to understand critical health information, to adult learners struggling with digital literacy, and to older adults trying to adapt to changes in today's information-centric society.

“传统”用户界面不断被移动，触摸和基于语音的互动范例所取代。这些帮助我们并赋予了我们权力。然而，移动接口也使许多脆弱的用户组边缘化。对于某些人来说，例如低素养的成年人或视力有限的用户，甚至简单的任务（例如，阅读书籍，了解用户手册，理解健康小册子）可能是一个斗争的目的地，智能，便携式和个人接口提供帮助的潜力。 For this, I propose to investigate and develop intelligent assistive technologies based on mobile, collaborative, and multimodal interfaces (speech, natural language, and gestural).******During my previous research I have extensively studioded the human factors of using imperfect Automatic Speech Recognition (ASR) and Natural Language Processing (NLP) systems, and have contributed to supporting natural interactions (such as gestures) with information-rich media and computing设备。我已经表明，如果非常适应用户的需求，则可以为边缘化的用户提供急需的帮助。反过来，当用户的需求和局限性告知其设计和功能时，启用此类多模式系统的基础技术可以改善其设计和功能。我计划继续遵循相同的方法结合交互设计和ASR/NLP研究。******该计划的目的是研究互动语音，语言和基于手势的移动辅助接口的方法和设计，这些互动辅助接口为用户提供实用的支持，以努力在努力执行诸如阅读，学习，学习和收集关键信息等基本活动。我计划研究多模式和协作，半自动化和交互式注释以及信息提取机制，这些机制将有助于用户从电子文档中获取基本知识。这项研究涵盖了广泛的语音和支持语言的媒体，将满足一系列潜水员的需求，从试图了解关键健康信息的低文字成年人到挣扎数字识字的成年学习者，以及试图适应当今信息以信息以信息为中心的社会的变化的老年人。