Collaborative Research: SHF: Small: Enabling Efficient 3D Perception: An Architecture-Algorithm Co-Design Approach

协作研究：SHF：小型：实现高效的 3D 感知：架构-算法协同设计方法

• 批准号: 2334624
• 负责人: Guoyu Lu
• 金额: $ 21.49万
• 依托单位: University of Georgia Research Foundation Inc
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-02-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2334624&HistoricalAwards=false
• 关键词: Collaborative; Research; SHF; Small; Enabling

This research project designs software-hardware collaborative mechanisms to boost the execution efficiency of 3D perception (point cloud) algorithms by an order of magnitude. Achieving this goal requires designing fundamentally new algorithmic primitives, offline and run-time systems, and hardware architectures that tame the irregular computation and memory patterns in 3D perception algorithms. This research unlocks next-generation software innovation in emerging domains driven by 3D perception, such as autonomous driving, Augmented/Virtual Reality, and precision agriculture. The research agenda is complemented by an educational/outreach agenda. The PIs are (1) offering summer “introduction to computing” courses to high school students from the Rochester Central School District, (2) engaging students in RIT’s National Technical Institute for the Deaf program through experiencing 3D sensing and research activities, (3) introducing new courses/modules in both UR and RIT on 3D perception, both on algorithms and hardware systems, and (4) offering undergraduate students inclusive opportunities for hands-on experience in emerging application domains and hardware acceleration.This research project addresses the fundamental mismatch between the irregularities in point-cloud algorithms and today’s hardware architectures, which are primarily optimized for 2D image- and video-processing algorithms that are regular stencil pipelines operating on structured data. The key intellectual merit is the pursuit of new algorithms and system architectures that reduce/eliminate irregular computation and memory accesses in 3D perception. The technical contribution is three-fold: 1) efficient, yet generally applicable, hardware building blocks required to accelerate point cloud algorithms, 2) run-time systems that dynamically adapt to operating constraints (e.g., hardware resources, performance, energy) in an application-aware and data-aware manner, and 3) a new class of efficient-by-construction 3D perception algorithms that leverage the small data volume in single-beam point clouds. The algorithm-hardware co-designed system not only accelerates current 3D perception algorithms, but also provides a computing substrate so that 3D perception can be pervasively used as a building block in future application domains.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该研究项目设计了软件-硬件协作机制，以将 3D 感知（点云）算法的执行效率提高一个数量级。实现这一目标需要设计全新的算法原语、离线和运行时系统以及驯服的硬件架构。这项研究解锁了由 3D 感知驱动的新兴领域的下一代软件创新，例如自动驾驶、增强/虚拟现实和精准农业。 PI 还辅以教育/推广议程：(1) 为罗切斯特中央学区的高中生提供夏季“计算机概论”课程，(2) 让学生通过体验 3D 参与 RIT 国家聋人技术学院项目。传感和研究活动，(3) 在 UR 和 RIT 中引入关于 3D 感知的新课程/模块，包括算法和硬件系统，以及 (4) 为本科生提供新兴应用领域和硬件实践经验的包容性机会该研究项目解决了点云算法与当今硬件架构之间的根本不匹配问题，这些算法主要针对二维图像和视频处理算法进行优化，这些算法是在结构化数据上运行的常规模板管道。追求新的算法和系统架构，以减少/消除 3D 感知中的不规则计算和内存访问。技术贡献有三方面：1）加速点云算法所需的高效且普遍适用的硬件构建块。 2) 以应用感知和数据感知的方式动态适应操作约束（例如硬件资源、性能、能源）的运行时系统，以及 3) 一种新型的高效构建 3D 感知算法单束点云中的小数据量不仅加速了当前的3D感知，而且还为未来3D感知可以普遍用作构建块算法提供了计算基础。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Object Detection Based on Raw Bayer Images

基于原始拜耳图像的物体检测

• 发表时间: 2023-10
• 期刊: 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS
• 作者: Lu; Guoyu
• 通讯作者: Guoyu

From Local to Holistic: Self-supervised Single Image 3D Face Reconstruction Via Multi-level Constraints

从局部到整体：通过多级约束进行自监督单图像 3D 人脸重建

• DOI: 10.1109/iros47612.2022.9982284
• 发表时间: 2022-10
• 期刊: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS
• 作者: Lu, Yawen;Sarkis, Michel;Bi, Ning;Lu, Guoyu
• 通讯作者: Lu, Guoyu

Multi-view Geometry Consistency Network for Facial Micro-Expression Recognition From Various Perspectives

多视角面部微表情识别的多视角几何一致性网络

• DOI: 10.1109/ijcnn55064.2022.9892565
• 发表时间: 2022-07
• 期刊: IEEE
• 作者: Parikh, Devarth;Lu, Yawen;Kasabov, Nikola;Lu, Guoyu
• 通讯作者: Lu, Guoyu