CHS: Small: Collaborative Research: Role-Based Norm Violation Response in Human-Robot Teams

CHS：小型：协作研究：人机团队中基于角色的规范违规响应

• 批准号: 1909847
• 负责人: Thomas Williams
• 金额: $ 50万
• 依托单位: Colorado School of Mines
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-10-01 至 2023-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1909847&HistoricalAwards=false
• 关键词: CHS; Small; Collaborative; Research; Role

Robots may need to carefully decide when and how to reject commands given to them, if the actions required to carry out those commands are not morally permissible. Most previous work on this topic takes a norm-based ethical approach, where a robot would operate under a set of rules describing what states or actions are morally wrong, and use those rules to explain its actions. In contrast, this project explores a role-based perspective, in which the robot reasons about the relationships it holds with others, the roles it plays in those relationships, and whether the actions requested of it are benevolent with respect to those roles and relationships. Specifically, the researchers will develop a framework to allow robots to reason in this way and generate explanations of its actions based on this reasoning. The researchers will then explore how role-based and norm-based command rejections compare in terms of how they affect human-robot teamwork, and design algorithms to allow robots to automatically decide what type of rejection to generate based on their context. These algorithms and explanations will be evaluated in two very different contexts with different types of relationships, roles, and rules: with civilian undergraduates at the Colorado School of Mines, and with Air Force cadets at the US Air Force Academy. This work will not only increase robots' ability to behave ethically and act as good teammates, but will also advance moral philosophy by providing experimental evidence for the relative importance and effectiveness of different tenets of role-based moral philosophy.More formally, the goals of this research are to investigate context-sensitive tradeoffs between rule-based and role-based responses, and the representations and mechanisms needed to facilitate role-based responses. The research team will do this by identifying metrics to assess response acceptability, quality, and effectiveness; modeling the generation of role-based responses and selection between role-based and rule-based responses; conducting experiments to validate those models and responses; and using the results to articulate novel moral and philosophical arguments. The team will start with exploratory studies at each experimental site contrasting the effectiveness of different command rejection phrasings formed by crossing different Speech-Act Theoretic communication strategies paired with different moral philosophical backgrounds, with respect to (a) field-standard survey measures of trust, likability, mindfulness, workload, and norm strength; (b) qualitative analysis of video data; and (c) statistical linguistic analyses from the multimodal interaction community. The researchers will then develop a framework for robots to generate these responses that will involve formal computational model development of inter-team relations, roles, and actions; machine learning-based modeling of norm violation response strategy selection using features such as task context characterization, role-theoretic proposed action benevolence, and expected responses to the responses; and integration into the DIARC robot cognitive architecture. Based on this work, the team will both advance traditional moral philosophical arguments and develop a novel framework in which moral philosophical arguments are justified based on the effects of computational models on moral psychology. Overall, the project will lead to foundational, interdisciplinary knowledge into norm violation response, consisting of algorithms for selecting between norm violation responses grounded in different ethical theories, guidelines for and insights into the design of morally competent language capable robots, novel computational accounts of role-based robot ethics, and novel empirically informed moral philosophical arguments.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

机器人可能需要仔细地决定何时以及如何拒绝给他们的命令，如果执行这些命令所需的措施在道德上不允许。 关于此主题的大多数以前的工作采用了一种基于规范的道德方法，在该方法中，机器人将根据一组规则进行操作，描述哪些状态或行动在道德上是错误的，并使用这些规则来解释其行动。相比之下，该项目探讨了一个基于角色的观点，其中机器人的原因是与他人之间的关系，其在这些关系中扮演的角色以及要求其所要求的行动是否对这些角色和关系的仁慈。具体来说，研究人员将开发一个框架，以使机器人以这种方式进行推理，并根据这种推理对其行为产生解释。 然后，研究人员将探讨如何基于角色和基于规范的命令拒绝如何与他们如何影响人类机器人团队合作以及设计算法相比，以允许机器人自动根据其上下文决定生成哪种类型的拒绝。这些算法和解释将在具有不同类型的关系，角色和规则的两个非常不同的情况下进行评估：科罗拉多州矿业学院的平民本科生，以及美国空军学院的空军学员。这项工作不仅将提高机器人的道德行为能力并充当好队友的能力，而且还将通过提供实验证据来提高道德哲学，以证明基于角色的道德哲学的不同原则的相对重要性和有效性。越来越正式地，这项研究的目标是研究基于规则的基于基础的和基于角色的反应和基于角色的反应和机构的依据，以及基于角色的响应以及基于角色的作用。 研究团队将通过确定指标来评估响应可接受性，质量和有效性来做到这一点；建模基于角色的响应的产生以及基于角色和基于规则的响应之间的选择；进行实验以验证这些模型和响应；并利用结果表达新颖的道德和哲学论点。 该团队将从每个实验地点的探索性研究开始，与（通过（a）野外标准的调查量度相对于（a）信任，可爱​​，正念，工作量和规范力量和规范的指标，与不同的语音效果理论交流策略形成了不同命令拒绝词的有效性； （b）视频数据的定性分析； （c）来自多模式相互作用社区的统计语言分析。然后，研究人员将为机器人开发一个框架，以生成这些响应，这些响应将涉及团队间关系，角色和行动的正式计算模型发展；基于机器学习的模型对规范违规响应策略选择诸如任务上下文特征，角色理论提出的行动仁慈以及对响应的预期响应等功能；并集成到DIARC机器人认知架构中。 基于这项工作，团队既将推进传统的道德哲学论点，又要开发一个新颖的框架，在该框架中，道德哲学论点是基于计算模型对道德心理学的影响而合理的。 总体而言，该项目将导致基础，跨学科的知识成为规范违规响应，包括算法，用于在不同的道德理论，指导方针，对具有道德能力的语言机器人设计的设计，基于角色的机器人伦理学的新颖计算说明以及基于经验性的道义哲学的新颖的道义宣传的诺斯特（Nep）的宣传。通过基金会的智力优点和更广泛的影响评估标准通过评估来支持。

项目成果

The Rule of Virtue: A Confucian Response to the Ethical Challenges of Technocracy

• DOI: 10.1007/s11948-021-00341-6
• 发表时间: 2021-10-01
• 期刊: SCIENCE AND ENGINEERING ETHICS
• 影响因子: 3.7
• 作者: Lan, Lishan;Zhu, Qin;Liu, Yongmou
• 通讯作者: Liu, Yongmou

Why and How Robots Should Say ‘No’

• DOI: 10.1007/s12369-021-00780-y
• 发表时间: 2021-05
• 期刊: International Journal of Social Robotics
• 影响因子: 4.7
• 作者: Gordon Briggs;T. Williams;R. Jackson;Matthias Scheutz

Comparing Norm-Based and Role-Based Strategies for Robot Communication of Role-Grounded Moral Norms

比较基于规范和基于角色的机器人通信基于角色的道德规范策略

• DOI: 10.1145/3571719
• 发表时间: 2023
• 期刊: ACM Transactions on Human-Robot Interaction
• 影响因子: 5.1
• 作者: Wen, Ruchen;Kim, Boyoung;Phillips, Elizabeth;Zhu, Qin;Williams, Tom
• 通讯作者: Williams, Tom

Robots as Moral Advisors: The Effects of Deontological, Virtue, and Confucian Role Ethics on Encouraging Honest Behavior

机器人作为道德顾问：义务论、美德和儒家角色伦理对鼓励诚实行为的影响

• DOI: 10.1145/3434074.3446908
• 发表时间: 2021
• 期刊: ACM/IEEE International Conference on Human-Robot Interaction
• 作者: Kim, Boyoung;Wen, Ruchen;Zhu, Qin;Williams, Tom;Phillips, Elizabeth
• 通讯作者: Phillips, Elizabeth

Unpretty Please: Ostensibly Polite Wakewords Discourage Politeness in both Robot-Directed and Human-Directed Communication

不漂亮的请：表面上礼貌的唤醒词不鼓励机器人主导和人类主导的沟通中的礼貌

• DOI: 10.1145/3536221.3556615
• 发表时间: 2022
• 期刊: International Conference on Multimodal Interaction
• 作者: Wen, Ruchen;Barton, Brandon;Fauré, Sebastian;Williams, Tom
• 通讯作者: Williams, Tom