CPS: Medium: Enabling Multimodal Sensing, Real-time Onboard Detection and Adaptive Control for Fully Autonomous Unmanned Aerial Systems

CPS：中：为完全自主的无人机系统实现多模态传感、实时机载检测和自适应控制

基本信息

批准号：
1739748
负责人：
Qinru Qiu
金额：
$ 40万
依托单位：
Syracuse University
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2017
资助国家：
美国
起止时间：
2017-10-01 至 2021-09-30
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1739748&HistoricalAwards=false
关键词：
CPS Medium Enabling Multimodal Sensing

项目摘要

The goal of this project is to investigate a low-cost and energy-efficient hardware and software system to close the loop between processing of sensor data, semantically high-level detection and trajectory generation in real-time. To safely integrate Unmanned Aerial Vehicles into national airspace, there is an urgent need to develop onboard sense-and-avoid capability. While deep neural networks (DNNs) have significantly improved the accuracy of object detection and decision making, they have prohibitively high complexity to be implemented on small UAVs. Moreover, existing UAV flight control approaches ignore the nonlinearities of UAVs and do not provide trajectory assurance. The research thrusts of this project are: (i) FPGA implementation of DNNs: both fully connected and convolutional layers of deep (convolutional) neural networks will be trained using (block-)circulant matrix and implemented using custom designed universal Fast Fourier Transform kernels on FPGA. This research thrust will enable efficient implementation of DNNs, reducing memory and computation complexity from O(N2) to O(N) and O(NlogN), respectively; (ii) autonomous detection and perception for onboard sense-and-avoid: existing regional detection neural networks will be extended to work with images taken from different angles, and multi-modal sensor inputs; (iii) real-time waypoint and trajectory generation - an integrated trajectory generation and feedback control scheme for steering under-actuated vehicles through desired waypoints in 3D space will be developed. For efficient implementation and hardware reuse, both detection and control problems will be formulated and solved using DNNs with (block-)circulant weight matrix. Deep reinforcement learning models will be investigated for waypoint generation and to assign artificial potential around the obstacles to guarantee a safe distance. The fundamental research results will enable onboard computing, real-time detection and control, which are cornerstones of autonomous and next-generation UAVs.

该项目的目标是研究一种低成本且节能的硬件和软件系统，以闭合传感器数据处理、语义高级检测和实时轨迹生成之间的循环。为了将无人机安全地融入国家空域，迫切需要发展机载感知和规避能力。虽然深度神经网络 (DNN) 显着提高了目标检测和决策的准确性，但在小型无人机上实施它们的复杂性却高得令人望而却步。此外，现有的无人机飞行控制方法忽略了无人机的非线性并且不提供轨迹保证。该项目的研究重点是：(i) DNN 的 FPGA 实现：深度（卷积）神经网络的全连接层和卷积层都将使用（块）循环矩阵进行训练，并使用定制设计的通用快速傅立叶变换内核来实现FPGA。这项研究主旨将实现 DNN 的高效实现，将内存和计算复杂度从 O(N2) 分别降低到 O(N) 和 O(NlogN)； (ii) 机载感知和回避的自主检测和感知：现有的区域检测神经网络将扩展到处理从不同角度拍摄的图像和多模式传感器输入； (iii) 实时航路点和轨迹生成——将开发一种集成轨迹生成和反馈控制方案，用于引导欠驱动车辆通过 3D 空间中的所需航路点。为了有效实施和硬件重用，检测和控制问题都将使用具有（块）循环权矩阵的 DNN 来制定和解决。将研究深度强化学习模型以生成路径点，并在障碍物周围分配人工势以保证安全距离。基础研究成果将实现机载计算、实时检测和控制，这是自主和下一代无人机的基石。

项目成果

期刊论文数量（33）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

Autonomous UAV with Learned Trajectory Generation and Control

具有学习轨迹生成和控制功能的自主无人机

DOI：
10.1109/sips47522.2019.9020508
发表时间：
2019-10
期刊：
IEEE Workshop on Signal Processing Systems (SIPS
影响因子：
0
作者：
Li, Yilan;Li, Mingyang;Sanyal, Amit;Wang, Yanzhi;Qiu, Qinru
通讯作者：
Qiu, Qinru

Finite-time Attitude Consensus Control of a Multi-Agent Rigid Body System

多智能体刚体系统的有限时间姿态一致控制

DOI：
10.23919/acc45564.2020.9147639
发表时间：
2020-07-01
期刊：
2020 American Control Conference (ACC)
影响因子：
0
作者：
Mohammad Maadani;E. Butcher;A. Sanyal
通讯作者：
A. Sanyal

Autonomous Waypoint Planning, Optimal Trajectory Generation and Nonlinear Tracking Control for Multi-rotor UAVs

多旋翼无人机自主航路点规划、最优轨迹生成和非线性跟踪控制

DOI：
10.23919/ecc.2019.8795855
发表时间：
2019-06-01
期刊：
2019 18th European Control Conference (ECC)
影响因子：
0
作者：
Hossein Eslamiat;Yilan Li;Ningshan Wang;A. Sanyal;Qinru Qiu
通讯作者：
Qinru Qiu

PatDNN: Achieving Real-Time DNN Execution on Mobile Devices with Pattern-based Weight Pruning

PatDNN：通过基于模式的权重修剪在移动设备上实现实时 DNN 执行

DOI：
10.1145/3373376.3378534
发表时间：
2020-01-01
期刊：
Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems
影响因子：
0
作者：
Wei Niu;Xiaolong Ma;Sheng Lin;Shihao Wang;Xuehai Qian;X. Lin;Yanzhi Wang;Bin Ren
通讯作者：
Bin Ren

Efficient Human Activity Classification from Egocentric Videos Incorporating Actor-Critic Reinforcement Learning

结合演员评论家强化学习，从以自我为中心的视频中进行有效的人类活动分类