S&AS: FND: COLLAB: Planning Coordinated Event Observation for Structured Narratives

S

• 批准号: 1849249
• 负责人: Dylan Shell
• 金额: $ 20万
• 依托单位: Texas A&M Engineering Experiment Station
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-03-15 至 2023-02-28
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1849249&HistoricalAwards=false
• 关键词: S& FND; COLLAB; Planning; Coordinated

People easily recognize the dramatic moments that unfold in human events. Dramatic turns of events are key to recognizing and communicating effective reports or stories about events. Autonomous systems will work more effectively with humans in obtaining and conveying such narrative when they too can recognize what is dramatic (or tragic, or comical) about human events. The challenge is to effectively convey such concepts to a computer in such a way that humans and autonomous systems can effectively work together in this. This research studies how to direct a team of robots to obtain video footage to produce clips that trace a dramatic story arc. It is an examination of how such systems might achieve goals that people consider to be abstract or high-level. Within this project, the programs that command teams of robots must predict likely events, direct the robots to be in position for obtaining the desired footage, and re-plan based on observed events. This challenge encompasses a rich and previously unstudied class of problems for robot systems. It will constitute a unique demonstration of robots that are capable of achieving high-level goals as they process data in forms which combine both continuous and discrete views of the world in a new and unusual way. More broadly, the research will advance how computers can fuse and summarize video streams. Both skills are needed for automatically generating synopses and in editing videos. Obvious places where this is useful include helping secure the nation (for surveillance), taming the deluge of online multimedia content (for summarization), and advancing applications in the creative industries (for editing). The research project will also use the ideas underlying these pieces in a new robotics course with students at three institutions going head-to-head in a series of competition-based class projects. This course (taught, among other places, at a Hispanic-Serving Institution) will contribute to the development of the STEM workforce of the future, helping increase American competitiveness.The project advances current knowledge by formulating new theory and developing novel algorithms for autonomous and robot systems, with a focus on those systems with minimal or no human operator intervention. The research contributes novel data representations for robots that will inhabit rich environments such as those characterized by uncertain, unanticipated, and dynamically changing circumstances. One of the foundational ideas of the project is a means to specify sophisticated mission objectives via a recursive structure using prior work in compiler theory for computer languages. The project involves a strong connection between this theoretical work and demonstrated systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

人们很容易识别人类事件中发生的戏剧性时刻。 事件的戏剧性转折是识别和传达有关事件的有效报告或故事的关键。 当自治系统也能够识别人类事件的戏剧性（或悲剧性或滑稽性）时，它们将更有效地与人类合作，获取和传达此类叙述。 挑战在于如何有效地将这些概念传达给计算机，以便人类和自主系统可以有效地协同工作。这项研究研究如何指导一组机器人获取视频片段，以制作追踪戏剧性故事情节的剪辑。它是对此类系统如何实现人们认为抽象或高级的目标的检验。在这个项目中，指挥机器人团队的程序必须预测可能发生的事件，指导机器人就位以获得所需的镜头，并根据观察到的事件重新计划。这一挑战涵盖了大量以前未研究过的机器人系统问题。 它将构成机器人的独特演示，这些机器人能够以一种新的、不寻常的方式结合连续和离散的世界观的形式处理数据，从而实现高水平的目标。 更广泛地说，这项研究将推进计算机如何融合和总结视频流。自动生成概要和编辑视频都需要这两种技能。这明显有用的地方包括帮助保护国家安全（用于监控）、控制在线多媒体内容的泛滥（用于总结）以及推进创意产业的应用（用于编辑）。该研究项目还将在新的机器人课程中使用这些作品的想法，三个机构的学生将在一系列基于竞赛的课程项目中进行正面交锋。这门课程（在拉美裔服务机构等地教授）将有助于未来 STEM 劳动力的发展，帮助提高美国的竞争力。该项目通过制定新理论和开发自主和自动化的新算法来推进当前知识。机器人系统，重点关注那些很少或没有人类操作员干预的系统。这项研究为机器人提供了新颖的数据表示，这些机器人将生活在丰富的环境中，例如那些具有不确定、不可预见和动态变化环境的环境。该项目的基本思想之一是使用计算机语言编译器理论中的先前工作，通过递归结构来指定复杂的任务目标。该项目涉及理论工作和演示系统之间的紧密联系。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Decision diagrams as plans: Answering observation-grounded queries

作为计划的决策图：回答基于观察的查询

• DOI: 10.1109/icra48891.2023.10161530
• 发表时间: 2023
• 期刊: IEEE
• 作者: Shell, Dylan A.;O'Kane, Jason M.
• 通讯作者: O'Kane, Jason M.

Charting the trade-off between design complexity and plan execution under probabilistic actions

绘制概率行为下设计复杂性和计划执行之间的权衡

• DOI: 10.1109/icra46639.2022.9811751
• 发表时间: 2022
• 期刊: IEEE
• 作者: Saberifar, Fatemeh Zahra;Shell, Dylan A.;O'Kane, Jason M.
• 通讯作者: O'Kane, Jason M.

Multiplexing Robot Experiments: Theoretical Underpinnings, Conditions for Existence, and Demonstrations

多重机器人实验：理论基础、存在条件和演示

• DOI: 10.1109/icra48506.2021.9561015
• 发表时间: 2021
• 期刊: IEEE International Conference on Robotics and Automation
• 作者: Moan, Rachel A.;Shell, Dylan A.;O'Kane, Jason M.
• 通讯作者: O'Kane, Jason M.

On nondeterminism in combinatorial filters

组合滤波器中的非确定性

• DOI: 10.1109/icra46639.2022.9812371
• 发表时间: 2022
• 期刊: IEEE
• 作者: Zhang, Yulin;Shell, Dylan A.
• 通讯作者: Shell, Dylan A.

A general class of combinatorial filters that can be minimized efficiently

一类可以有效最小化的通用组合滤波器

• DOI: 10.1109/icra48891.2023.10160479
• 发表时间: 2023
• 期刊: IEEE
• 作者: Zhang, Yulin;Shell, Dylan A.
• 通讯作者: Shell, Dylan A.