CRII: RI: Joint Models of Language and Context for Robotic Language Acquisition

CRII：RI：机器人语言习得的语言和语境联合模型

• 批准号: 1657469
• 负责人: Cynthia Matuszek
• 金额: $ 16.31万
• 依托单位: University of Maryland Baltimore County
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-08-01 至 2020-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1657469&HistoricalAwards=false
• 关键词: CRII; RI; Joint; Models; Language

As robots become smaller, less expensive, and more capable, they are able to perform an increasing variety of tasks, leading to revolutionary improvements in domains such as automobile safety and manufacturing. However, their inflexibility makes them hard to deploy in human-centric environments such as homes and schools, where their tasks and environments are constantly changing. Meanwhile, learning to understand language about the physical world is a growing research area in both robotics and natural language processing. The core problem is how the meanings of words are grounded in the noisy, perceptual world in which a robot operates. This project explores how robots can learn about the world from natural language in order to take instructions and learn about their environment naturally and intuitively from people. The ability to follow directions reduces the adoption barrier for robots in domains such as assistive technology, education, and caretaking, where interactions with non-specialists are crucial. Such robots have the potential to ultimately improve autonomy and independence for populations such as aging-in-place elders; for example, a manipulator arm that can learn from a user?s explanation how to handle food or open novel containers would directly affect the independence of persons with dexterity concerns such as advanced arthritis. This is an exploratory investigation of how linguistic and perceptual models can be expanded during interaction, allowing robots to understand novel language about unanticipated domains. In particular, the focus is on developing new learning approaches that correctly induce joint models of language and perception, building data-driven language models that add new semantic representations over time. The work combines semantic parser learning, which provides a distribution over possible interpretations of language, with perceptual representations of the underlying world. New concepts are added on the fly as new words and new perceptual data are encountered, and a semantically meaningful model can be trained by maximizing the expected likelihood of language and visual components. This integrated approach allows for effective model updates with no explicit labeling of words or percepts. This approach will be combined with experiments on improving learning efficiency by incorporating active learning, leveraging a robot's ability to ask questions about objects in the world.

随着机器人变得越来越小，便宜且功能更高，他们能够执行越来越多的任务，从而导致诸如汽车安全和制造等领域的革命性改进。但是，它们的僵硬性使他们难以在以人为中心的环境（例如房屋和学校）中部署，在这些环境中，他们的任务和环境在不断变化。同时，学习了解物理世界的语言是机器人技术和自然语言处理中不断增长的研究领域。核心问题是单词的含义如何基于机器人运行的嘈杂，感知世界。该项目探讨了机器人如何从自然语言中学习世界，以便从人自然和直观地了解他们的环境。遵循指示的能力降低了辅助技术，教育和照料等领域的机器人的采用障碍，与非专家的互动至关重要。这样的机器人有可能最终提高诸如现有长老等人口的自主权和独立性。例如，一个可以从用户那里学习的操纵臂如何处理食物或开放的新颖容器将直接影响具有敏捷问题（例如高级关节炎）的人的独立性。这是对如何在互动过程中扩展语言和感知模型的探索性研究，使机器人可以理解有关意外领域的新语言。特别是，重点是开发正确诱导语言和感知联合模型的新学习方法，构建数据驱动的语言模型，以随着时间的推移添加新的语义表示。该作品结合了语义解析器的学习，该学习提供了对语言可能解释的分布，以及对基础世界的感知表示。随着新单词和新的感知数据，可以随时添加新概念，并且可以通过最大化语言和视觉组件的预期可能性来训练语义上有意义的模型。这种集成的方法允许有效的模型更新，而无明确的单词或知觉标记。这种方法将通过合并积极的学习，利用机器人提出有关世界上对象的问题的能力来提高学习效率的实验。

项目成果

Unsupervised Selection of Negative Examples for Grounded Language Learning

扎根语言学习的反面例子的无监督选择

• 发表时间: 2018
• 期刊: Proceedings of the 32nd Conference on Artificial Intelligence (AAAI
• 作者: Pillai, Nisha;Matuszek, Cynthia
• 通讯作者: Matuszek, Cynthia

Neural Variational Learning for Grounded Language Acquisition

• DOI: 10.1109/ro-man50785.2021.9515374
• 发表时间: 2021-07
• 期刊: 2021 30th IEEE International Conference on Robot & Human Interactive Communication (RO-MAN)
• 作者: Nisha Pillai;Cynthia Matuszek;Francis Ferraro

Grounded Language Learning: Where Robotics and NLP Meet (invited talk)

扎根语言学习：机器人学和 NLP 的交汇点（特邀演讲）

• 发表时间: 2018
• 期刊: Proceedings of the International Joint Conference on Artificial Intelligence
• 作者: Matuszek, Cynthia

Sampling Approach Matters: Active Learning for Robotic Language Acquisition

• DOI: 10.1109/bigdata50022.2020.9378415
• 发表时间: 2020-11
• 期刊: 2020 IEEE International Conference on Big Data (Big Data)
• 作者: Nisha Pillai;Edward Raff;Francis Ferraro;Cynthia Matuszek

Learning Object Attributes with Category-Free Grounded Language from Deep Featurization

• DOI: 10.1109/iros45743.2020.9340824
• 发表时间: 2020-10
• 期刊: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
• 作者: Luke E. Richards;Kasra Darvish;Cynthia Matuszek