Towards next-generation context-aware affective human-machine interfaces

迈向下一代情境感知情感人机界面

• 批准号: RGPIN-2016-04175
• 负责人: Falk, Tiago
• 金额: $ 3.28万
• 依托单位: Institut national de la recherche scientifique
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2020
• 资助国家: 加拿大
• 起止时间: 2020-01-01 至 2021-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=708239
• 关键词: Towards; next; generation; context; aware

Until recently, the ubiquitous way of interacting with digital information was via a keyboard and mouse. As mobile devices gained popularity, however, touch became the primary input modality for human-machine interfaces (HMIs), followed closely by voice. The next decade, however, will witness tremendous growth in data generation (e.g., statistics suggest a 50-fold increase in our digital footprint by 2020). New sensors and technologies will emerge and will be embedded in everyday objects, creating the so-called Internet-of-Things revolution. This data deluge will require drastic innovations in human-machine interaction, particularly as the boundaries between humans and machines, as well as machines and the environment become blurred. To achieve this ambitious goal, the proposed research program aims at developing technologies that will enhance machine intelligence by making them not only aware of their surroundings, but also of their users' affective/cognitive states. Such context-aware affective HMIs will revolutionize several key sectors of the Canadian economy, including healthcare, entertainment, education, and telecommunications. To achieve this long-term goal, six short-term objectives have been defined, grouped under three main themes. First, we propose to take existing context-aware solutions to the next level by i) enabling distributed environment awareness, ii) shifting the human-machine boundary by making the body as an input modality, and iii) giving the machine additional intelligence via an innovative Quality-of-Context metric. Voice and neurophysiological input modalities, however, are extremely sensitive to noise (e.g., room acoustics) and artefacts (e.g., due to movement), respectively. To overcome this severe usability factor, the second proposed theme aims at developing next-generation context-aware enhancement algorithms for both iv) speech and v) neurophysiological signals. Lastly, we propose to vi) build a fully-functional context-aware prototype that uses voice, gesture, and body as HMI input control signals. Validation experiments within a robot control application will be implemented, thus opening doors for longer-term applications, such as telemedicine robots that are aware of the patient's mental state and could, for example, detect the onset of depression and take necessary measures accordingly. The proposed research program will place emphasis on excellent training of highly qualified personnel (HQP). HQP will be exposed to an interdisciplinary melange of signal processing, cognitive engineering, machine learning, and hands-on prototyping, as well as to research facilities that feature state-of-the-art wearable technologies and environment monitoring sensors. In addition, via close collaboration with industry, HQP will be equipped with skills and expertise that are in high demand in today's competitive job market.

直到最近，与数字信息交互的普遍方式还是通过键盘和鼠标。然而，随着移动设备的普及，触摸成为人机界面 (HMI) 的主要输入方式，其次是语音。然而，未来十年将见证数据生成的巨大增长（例如，统计数据表明，到 2020 年，我们的数字足迹将增加 50 倍）。新的传感器和技术将会出现并嵌入到日常物品中，从而引发所谓的物联网革命。这种数据洪流将需要人机交互方面的重大创新，特别是当人与机器、机器与环境之间的界限变得模糊时。为了实现这一雄心勃勃的目标，拟议的研究计划旨在开发增强机器智能的技术，使机器不仅了解周围环境，还了解用户的情感/认知状态。这种情境感知的情感人机界面将彻底改变加拿大经济的几个关键领域，包括医疗保健、娱乐、教育和电信。 为了实现这一长期目标，确定了六个短期目标，分为三个主题。首先，我们建议通过以下方式将现有的上下文感知解决方案提升到一个新的水平：i）实现分布式环境感知，ii）通过将身体作为输入方式来改变人机边界，以及iii）通过创新的上下文质量指标。然而，语音和神经生理学输入方式分别对噪声（例如，室内声学）和伪影（例如，由于运动）极其敏感。为了克服这一严重的可用性因素，第二个提出的主题旨在开发下一代上下文感知增强算法，用于 iv) 语音和 v) 神经生理信号。最后，我们建议 vi) 构建一个功能齐全的上下文感知原型，使用语音、手势和身体作为 HMI 输入控制信号。将在机器人控制应用程序中进行验证实验，从而为长期应用打开大门，例如远程医疗机器人可以了解患者的精神状态，例如可以检测抑郁症的发作并相应地采取必要的措施。 拟议的研究计划将重点放在高素质人才（HQP）的卓越培训上。 HQP 将接触信号处理、认知工程、机器学习和动手原型制作的跨学科融合，以及采用最先进的可穿戴技术和环境监测传感器的研究设施。此外，通过与业界的密切合作，总部将具备当今竞争激烈的就业市场急需的技能和专业知识。