CAREER: Robots, Speech, and Learning in Inclusive Human Spaces

职业：包容性人类空间中的机器人、语音和学习

• 批准号: 2145642
• 负责人: Cynthia Matuszek
• 金额: $ 54.89万
• 依托单位: University of Maryland Baltimore County
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-06-01 至 2027-05-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2145642&HistoricalAwards=false
• 关键词: CAREER; Robots; Speech; Learning; Inclusive

As robots become more capable and ubiquitous, they are increasingly moving into traditionally human-centric environments such as as health care, education, and elder care. As robots engage in tasks as diverse as helping with household work, deploying medication, and tutoring students, it becomes increasingly critical for them to interact naturally with the people around them. Key to this progress is the development of robots that acquire an understanding of goals and objects from natural communications with a diverse set of end users. One way to address this is using language to build systems that learn from people they are interacting with. Algorithms and systems developed in this project will allow robots to learn about the world around them from linguistic interactions. This research will focus on understanding spoken language about the physical world from diverse groups of people, resulting in systems that are more able to robustly handle a wide variety of real-world interactions. Ultimately, the project will increase the usability and fairness of robots deployed in human spaces.This CAREER project will study how robots can learn about noisy, unpredictable human environments from spoken language combined with perception, using context derived from sensors to constrain the learning problem. Grounded language refers to language that occurs in and refers to the physical world in which robots operate. Human interactions are fundamentally contextual: when learning about the world, we focus learning by considering not only direct communication but also the context of that interaction. For much existing work on learning to understand physically situated language, text is the primary interlingua, and context is considered relatively narrowly. Additionally, reliance on pre-existing large datasets has begun to raise questions about bias and inclusivity in learning-driven technologies. To address these limitations, this work will focus on learning semantics directly from perceptual inputs combined with speech from diverse sources. The goal is to develop learning infrastructure, algorithms, and approaches to enable robots to learn to understand task instructions and object descriptions from spoken communication with end users. The project will develop new methods of efficiently learning from multi-modal data inputs, with the ultimate goal of enabling robots to efficiently and naturally learn about their world and the tasks they should perform.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

随着机器人变得越来越有能力和无处不在，它们越来越多地进入传统上以人为中心的环境，例如医疗保健，教育和老年护理。随着机器人从事多样化的任务，例如帮助家庭工作，部署药物和辅导学生，他们与周围的人自然互动变得越来越重要。进度的关键是机器人的开发，这些机器人从自然通信中与各种最终用户的自然通信中获得了了解。解决此问题的一种方法是使用语言来构建从他们正在与之互动的人那里学习的系统。该项目中开发的算法和系统将使机器人可以从语言互动中了解周围的世界。这项研究将着重于了解各种各样的人群的物理世界的口语，从而导致系统更有能力处理各种现实世界的互动。最终，该项目将增加在人类空间中部署的机器人的可用性和公平性。本职业项目将研究机器人如何利用传感器来限制学习问题的环境，从口头语言中学习嘈杂，不可预测的人类环境。扎根的语言是指发生在机器人操作的物理世界中的语言。人类互动在根本上是背景：当学习世界时，我们不仅要考虑直接交流，而且考虑这种互动的背景来集中学习。对于学习理解物理位置语言的许多现有工作，文本是主要的interlingua，并且上下文被认为相对狭窄。此外，依赖预先存在的大型数据集的依赖已开始引发有关学习驱动技术中偏见和包容性的问题。为了解决这些局限性，这项工作将集中于直接从感知输入以及各种来源的语音的学习语义上。目的是开发学习基础架构，算法和方法，使机器人能够从与最终用户进行交流中了解任务说明和对象描述。该项目将开发新的方法，从多模式数据输入中有效学习，最终的目标是使机器人能够有效自然地了解其世界以及他们应该执行的任务。该奖项反映了NSF的法定任务，并通过评估该基金会的知识分子功能和广泛的影响来审查Criteria。

项目成果

Scarecrows in Oz: The Use of Large Language Models in HRI

奥兹国的稻草人：大型语言模型在 HRI 中的使用

• 发表时间: 2024
• 期刊: ACM transactions on humanrobot interaction
• 作者: Williams, Tom;Matuszek, Cynthia;Mead, Ross;DePalma, Nick
• 通讯作者: DePalma, Nick

Voice in the Machine: Ethical Considerations for Language-Capable Robots

机器中的声音：具有语言能力的机器人的道德考虑

• 发表时间: 2023
• 期刊: Communications of the ACM
• 影响因子: 22.7
• 作者: Wiliams, Tom;Matuszek, Cynthia;Jokinen, Kristiina;Korpan, Raj;Pustejovsky, James;Scassellati, Brian
• 通讯作者: Scassellati, Brian

Measuring Equality in Machine Learning Security Defenses: A Case Study in Speech Recognition

• DOI: 10.1145/3605764.3623911
• 发表时间: 2023-02
• 期刊: Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security
• 作者: Luke E. Richards;Edward Raff;Cynthia Matuszek

Lessons From A Small-Scale Robot Joining Experiment in VR

小型机器人参与 VR 实验的经验教训

• 发表时间: 2023
• 期刊: and Mixed-Reality for Human-Robot Interactions (VAM-HRI
• 作者: Higgins, Padraig;Barron, Ryan;Engel, Don;Matuszek, Cynthia
• 通讯作者: Matuszek, Cynthia

Machine Learning Security as a Source of Unfairness in Human-Robot Interaction

机器学习安全是人机交互不公平的根源

• 发表时间: 2023
• 期刊: and Community (DEI HRI
• 作者: Richards, Luke E.;Matuszek, Cynthia
• 通讯作者: Matuszek, Cynthia