CHS:Small: A Kinder, Gentler Technology: Enhancing Human-Machine Symbiosis Using Adaptive, Personalized Affect-Aware Systems

CHS：Small：更友善、更温和的技术：使用自适应、个性化情感感知系统增强人机共生

• 批准号: 1717705
• 负责人: Vesna Novak
• 金额: $ 44.79万
• 依托单位: University of Wyoming
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-08-15 至 2020-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1717705&HistoricalAwards=false
• 关键词: CHS; Small; Kinder; Gentler; Technology

A longstanding goal in artificial intelligence is to develop smart systems that interact well with humans. Advances in sensing and machine learning are increasingly allowing computers to infer mental states, raising questions about how agents might use those inferences to adapt to human partners. This project will systematically address how to design and evaluate "affect-aware" systems that adapt their behavior based on estimates of their users' emotional experiences. The team will first look at the effectiveness of current strategies that vary the difficulty of educational tasks and games based on inferred affect. They will then develop new strategies that take into account both individual personality and dynamic characteristics of the physical environment. Finally, they will evaluate these strategies, paying particular attention to what happens when systems act on incorrect inferences about affect. These studies will help pave the way toward self-driving cars, conversational assistants, and virtual reality characters that consider affect when interacting with people, ideally leading to better experiences and outcomes. The team will also develop new interdisciplinary courses in human factors and human-computer interaction, connecting with industrial partners to help train students in both the practice and research of these kinds of adaptive systems. Further, they will do public outreach about these systems and use them to provide summer research experiences for K-12 and community college students, focusing on those from groups traditionally underrepresented in computing.The project will be structured as a series of lab studies, using spatial cognition games and robot-assisted motor rehabilitation tasks as testbeds that allow the team to directly manipulate task difficulty and measure enjoyment/engagement and performance/learning outcomes. The team will first collect training data with people using the testbeds at randomly selected difficulty levels and reporting the perceived level of difficulty as too easy (bored), too hard (frustrated), or about right, while capturing heart rate signals, skin conductance and temperature, electroencephalogram (EEG) data, and environmental factors including light, time of day, and room temperature. These will be used to train affect recognizers using a variety of machine learning methods: linear discriminant analysis (including a Kalman adaptive version), support vector machines, neural and Bayesian networks, and random forests. Using a common adaptation strategy that adjusts difficulty up or down one step, the team will measure the enjoyment and performance outcomes that affect-aware recognizers achieve both with and without considering environmental factors, comparing those to a baseline strategy that adapts difficulty based only on task performance. During these experiments, the team will also collect data about users' personality characteristics and use those to develop individualized recognition models and adaptation strategies for different personality types. These individualized models and strategies will be evaluated by comparing them to the baseline data from the first experiment. Finally, they will compare the outcomes of these systems with those from a "best-case" system controlled by humans and a "worst-case" error-prone system that chooses adaptation strategies randomly, looking at those induced error rates along with the natural error rates captured during the other experiments to determine the effect of recognition and adaptation error on satisfaction and task outcomes.

人工智能中的一个长期目标是开发与人类互动良好的智能系统。 传感和机器学习的进步越来越多地允许计算机推断精神状态，提出了有关代理如何使用这些推论来适应人类伴侣的问题。 该项目将系统地介绍如何根据用户的情感体验估计来设计和评估“情感感知”系统，以适应其行为。 该团队将首先研究基于推论的影响，这些策略的有效性改变了教育任务和游戏的难度。 然后，他们将制定新的策略，这些策略考虑到个人个性和身体环境的动态特征。 最后，他们将评估这些策略，特别关注系统对情感的不正确推论时发生的情况。 这些研究将有助于为自动驾驶汽车，对话助手和虚拟现实特征铺平道路，这些方式在与人互动时考虑影响，理想地带来了更好的体验和结果。 该团队还将在人为因素和人为计算机互动方面开发新的跨学科课程，与工业伙伴建立联系，以帮助培训学生对这些自适应系统的实践和研究。 此外，他们将对这些系统进行公众的宣传，并使用它们为K-12和社区大学生提供夏季研究经验，重点关注传统上代表性不足的小组的研究经验。该项目将以一系列实验室研究的形式构建，使用空间认知游戏和机器人辅助康复任务，可以将团队作为测试床，从而可以直接学习和努力进行努力和努力进行努力/互动。 该团队将首先与人们使用测试床以随机选择的难度水平收集培训数据，并将感知到的难度水平报告为太容易（无聊），太硬（沮丧）或正确，同时捕获心率信号，皮肤电导和温度，电脑电脑电导率和电脑电脑图（EEG）数据，以及环境因素，包括光线，时间和室温。 这些将用于使用各种机器学习方法来训练识别者：线性判别分析（包括卡尔曼自适应版本），支持向量机，神经和贝叶斯网络以及随机森林。 使用一种常见的适应策略，该策略可以调整一个步骤或向下一步，团队将衡量影响感知意识识别者在有没有考虑环境因素的情况下实现的享受和绩效成果，将这些成果与基准策略进行比较，该策略仅根据任务绩效调整难度。 在这些实验中，团队还将收集有关用户性格特征的数据，并使用这些数据来开发个性化的识别模型和针对不同人格类型的适应策略。这些个性化模型和策略将通过将它们与第一个实验的基线数据进行比较来评估。 最后，他们将将这些系统的结果与由人类控制的“最佳案例”系统和“最糟糕的”系统控制的系统进行比较，该系统和“最糟糕的”错误系统选择了随机选择适应策略，查看这些系统的错误率以及其他实验中捕获的自然错误率，以确定识别和适应性误差对满意度和任务的影响。

项目成果

A New Method for Classification of Hazardous Driver States Based on Vehicle Kinematics and Physiological Signals

• DOI: 10.1007/978-3-030-11051-2_10
• 发表时间: 2019-02
• 作者: Mickael Aghajarian;A. Darzi;J. McInroy;D. Novak

A Brief Measure of Interpersonal Interaction for 2-Player Serious Games: Questionnaire Validation

• DOI: 10.2196/12788
• 发表时间: 2019-07-01
• 期刊: JMIR SERIOUS GAMES
• 影响因子: 4
• 作者: Gorsic, Maja;Clapp, Joshua D.;Novak, Domen
• 通讯作者: Novak, Domen

Using Physiological Linkage for Patient State Assessment In a Competitive Rehabilitation Game

• DOI: 10.1109/icorr.2019.8779361
• 发表时间: 2019-01-01
• 期刊: 2019 IEEE 16TH INTERNATIONAL CONFERENCE ON REHABILITATION ROBOTICS (ICORR)
• 作者: Darzi, Ali;Novak, Domen
• 通讯作者: Novak, Domen

Classification of Multiple Psychological Dimensions in Computer Game Players Using Physiology, Performance, and Personality Characteristics

• DOI: 10.3389/fnins.2019.01278
• 发表时间: 2019-11-26
• 期刊: FRONTIERS IN NEUROSCIENCE
• 影响因子: 4.3
• 作者: Darzi, Ali;Wondra, Trent;Novak, Domen
• 通讯作者: Novak, Domen

Cooperative Cooking: A Novel Virtual Environment for Upper Limb Rehabilitation

• DOI: 10.1109/embc.2018.8513005
• 发表时间: 2018-01-01
• 期刊: 2018 40TH ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY (EMBC)
• 作者: Gorsic, Maja;Ha Tran, Minh;Novak, Domen
• 通讯作者: Novak, Domen