SHF: Small: Collaborative Research: Software-Defined Imaging for Energy-Efficient Visual Computing

SHF：小型：协作研究：用于节能视觉计算的软件定义成像

• 批准号: 1909663
• 负责人: Suren Jayasuriya
• 金额: $ 33.3万
• 依托单位: Arizona State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1909663&HistoricalAwards=false
• 关键词: SHF; Small; Collaborative; Research; Software

Image sensors are becoming ubiquitous in daily life as they are incorporated in future intelligent systems including autonomous navigation, health monitoring, and robotics. A central challenge in these camera-driven applications is the inflexibility of current sensor designs and their consequent energy cost. This project designs a new category of image sensors which exploit hardware -- software co-design to attain better sensing at lower cost. The project advances a vertically-integrated design from the mixed-signal sensor circuitry to the computational architecture and the operating system software support. The project's impacts are the creation of new, flexible image sensor systems that can be used for a variety of visual computing applications. The project further seeks to include broadening access to education and research through curriculum material which emphasize smart cameras of the future, outreach to middle and high school students in a summer program to discover imaging applications, and industry engagement through workshops on software-defined imaging. The project focuses on designing software-defined image sensors, which offer new dimensions of configurability along with system support and programming abstractions to support application-specific needs. To achieve this, the project focuses on three main objectives: (1) Design and implementation of configurable sensor primitives, including programmable regions of interest with custom exposure, readout, and quantization, (2) architectures to control these sensor primitives, and to accelerate image signal processing and vision workloads, and (3) operating system services for scheduling the memory needs for our new sensor primitives, and OS interfaces that enable low-latency reactive sensor control for key applications. These innovations are evaluated in an integrative evaluation testbed that includes a fabricated software-defined image sensor prototype and a Field Programmable Gate Array (FPGA)-based system to measure energy and performance for a set of end-to-end visual applications.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

图像传感器在日常生活中变得无处不在，因为它们被纳入未来的智能系统，包括自主导航、健康监测和机器人技术。这些相机驱动应用的一个主要挑战是当前传感器设计的不灵活性及其随之而来的能源成本。该项目设计了一种新型图像传感器，它利用硬件-软件协同设计以更低的成本获得更好的传感效果。该项目推进了从混合信号传感器电路到计算架构和操作系统软件支持的垂直集成设计。该项目的影响是创建新的、灵活的图像传感器系统，可用于各种视觉计算应用。该项目还寻求通过强调未来智能相机的课程材料扩大教育和研究的机会，在暑期项目中向中学生和高中生推广以发现成像应用，并通过软件定义成像研讨会进行行业参与。该项目的重点是设计软件定义的图像传感器，它提供了新的可配置性维度以及系统支持和编程抽象，以支持特定于应用程序的需求。为了实现这一目标，该项目重点关注三个主要目标：(1) 设计和实现可配置的传感器基元，包括具有自定义曝光、读出和量化功能的可编程感兴趣区域，(2) 控制这些传感器基元并加速的架构图像信号处理和视觉工作负载，以及 (3) 用于调度新传感器原语内存需求的操作系统服务，以及为关键应用程序实现低延迟反应式传感器控制的操作系统接口。这些创新在综合评估测试台中进行评估，该测试台包括制造的软件定义图像传感器原型和基于现场可编程门阵列 (FPGA) 的系统，用于测量一组端到端视觉应用的能量和性能。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Adaptive Resolution-Based Tradeoffs for Energy-Efficient Visual Computing Systems

节能视觉计算系统的基于自适应分辨率的权衡

• DOI: 10.1109/mprv.2021.3052528
• 发表时间: 2021-04
• 期刊: IEEE Pervasive Computing
• 影响因子: 1.6
• 作者: LiKamWa, Robert;Hu, Jinhan;Kodukula, Venkatesh;Liu, Yifei
• 通讯作者: Liu, Yifei

Rhythmic pixel regions: multi-resolution visual sensing system towards high-precision visual computing at low power

节奏像素区域：低功耗高精度视觉计算的多分辨率视觉传感系统

• DOI: 10.1145/3445814.3446737
• 发表时间: 2021-04-19
• 期刊: Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems
• 作者: Venkatesh Kodukula;Ale;er Shearer;er;Van Nguyen;Srinivas Lingutla;Yifei Liu;R. Likamwa
• 通讯作者: R. Likamwa

Energy-Efficient Object Tracking Using Adaptive ROI Subsampling and Deep Reinforcement Learning

使用自适应 ROI 子采样和深度强化学习的节能对象跟踪

• DOI: 10.1109/access.2023.3270776
• 发表时间: 2024-09-13
• 期刊: IEEE Access
• 影响因子: 3.9
• 作者: Sameeksha Katoch;Odrika Iqbal;A. Spanias;Suren Jayasuriya
• 通讯作者: Suren Jayasuriya

Design and FPGA Implementation of an Adaptive Video Subsampling Algorithm for Energy-Efficient Single Object Tracking

用于节能单目标跟踪的自适应视频子采样算法的设计和 FPGA 实现

• 发表时间: 2020-01
• 期刊: IEEE International Conference on Image Processing
• 作者: Iqbal, Odrika;Siddiqui, Saquib;Martin, Joshua;Katoch, Sameeksha;Spanias, Andreas;Bliss, Daniel;Jayasuriya, Suren
• 通讯作者: Jayasuriya, Suren

Software-Defined Imaging: A Survey

软件定义成像：调查

• DOI: 10.1109/jproc.2023.3266736
• 发表时间: 2023-05-01
• 期刊: Proceedings of the IEEE
• 影响因子: 20.6
• 作者: Suren Jayasuriya;Odrika Iqbal;Venkatesh Kodukula;Victor Torres;R. Likamwa;A. Spanias
• 通讯作者: A. Spanias