CSR: Small: Reconfigurable In-Sensor Architectures for High Speed and Low Power In-situ Image Analysis

CSR：小型：可重构传感器内架构，用于高速、低功耗原位图像分析

• 批准号: 1618606
• 负责人: Christophe Bobda
• 金额: $ 47.79万
• 依托单位: University of Arkansas
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-10-01 至 2019-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1618606&HistoricalAwards=false
• 关键词: CSR; Small; Reconfigurable; Sensor; Architectures

Cameras are pervasively used for surveillance and monitoring applications and can capture a substantial amount of image data. The processing of this data, however, is either performed a posteriori or at powerful backend servers. While a posteriori and non-real-time video analysis may be sufficient for certain groups of applications, it does not suffice for applications such as autonomous navigation in complex environments, or hyper spectral image analysis using cameras on drones, that require near real-time video and image analysis, sometimes under SWAP (Size Weight and Power) constraints. This work hypothesizes that future data challenges in real-time imaging can be overcome by pushing computation into the image sensor. Such systems will exploit the massive parallel nature of sensor arrays to reduce the amount of data analyzed at the processing unit. To this end, vertically integrated technology, such as focal plane sensor processors (FPSP), have been developed to overcome the limitations of conventional image processing systems. While some of these devices are programmable and offer the benefits of close-to-sensor processing such as performance and bandwidth reduction, they exhibit many drawbacks. For instance, each column of pixels is handled by a single processor, which reduces the parallelism and all pixels are treated equally and processed at the same rate, despite differences in input relevance for the application at hand. Consequently, systems spend more time spinning on non-relevant data, which increases sensing and computation time and power consumption. Research on FPSPs has mostly focused on technology aspects with some proof of concepts. Architectural design approaches, that involve high-level synthesis with the goal of mapping applications to low-level architectures, have not gained a lot of attention.To overcome the limitations of existing architectures, the goal of this research is the design of a highly parallel, hierarchical, reconfigurable and vertically-integrated 3D sensing-computing architecture (XPU), along with high-level synthesis methods for real-time, low-power video analysis. The architecture is composed of hierarchical intertwined planes, each of which consists of computational units called XPUs. The lowest-level plane processes pixels in parallel to determine low level shapes in an image while higher-level planes use outputs from low-level planes to infer global features in the image. The proposed architecture presents three novel contributions: a hierarchical, configurable architecture for parallel feature extraction in video streams, a machine learning based relevance-feedback method that adapts computational performance and resource usage to input data relevance, and a framework for converting sequential image processing algorithms to multiple layers of parallel computational processing units in the sensor. The results of this projects can be used in other fields, where large amounts of processing need to be performed on data collected by generic sensors deployed in the field. Furthermore, mechanisms for translating sequential constructs into functionally equivalent accelerators using hardware constructs will lead to highly parallel and efficient sensing units that can perform domain specific tasks more efficiently.

相机普遍地用于监视和监视应用程序，并可以捕获大量的图像数据。但是，该数据的处理要么是在强大的后端服务器上执行的。虽然后验和非实时视频分析可能足以满足某些应用程序组，但对于诸如复杂环境中的自主导航或使用无人机上的摄像机进行超级频谱图像分析之类的应用不足，需要接近实时的视频和图像分析，有时在交换（尺寸重量和功率）约束下。这项工作假设可以通过将计算推向图像传感器来克服实时成像中未来的数据挑战。这样的系统将利用传感器阵列的巨大平行性质，以减少处理单元分析的数据量。为此，已经开发了垂直集成的技术，例如焦平面传感器处理器（FPSP），以克服传统图像处理系统的局限性。尽管其中一些设备是可编程的，并提供了近传感器处理的好处，例如性能和降低带宽，但它们表现出许多缺点。例如，每列的像素的每一列均由单个处理器处理，该处理器降低了并行性，尽管输入相关性差异与手头应用的输入相关性差异，但所有像素都得到了平等处理并以相同的速率处理。因此，系统花费更多的时间在非相关数据上旋转，从而增加了传感和计算时间和功耗。 FPSP的研究主要集中在具有一些概念证明的技术方面。 Architectural design approaches, that involve high-level synthesis with the goal of mapping applications to low-level architectures, have not gained a lot of attention.To overcome the limitations of existing architectures, the goal of this research is the design of a highly parallel, hierarchical, reconfigurable and vertically-integrated 3D sensing-computing architecture (XPU), along with high-level synthesis methods for real-time,低功耗视频分析。该体系结构由层次相互交织的平面组成，每个平面由称为XPU的计算单元组成。最低级平面的过程平行于确定图像中的低级别形状，而高级平面使用低级平面的输出来推断图像中的全局特征。所提出的体系结构提出了三个新颖的贡献：用于视频流中并行特征提取的层次结构架构，一种基于机器学习的相关性反馈方法，可将计算性能和资源用法调整以输入数据相关性，以及用于将传感器中并行计算处理单元的多层转换为传感器中的多层图像处理算法的框架。该项目的结果可用于其他领域，在其他领域，需要对部署在该字段中的通用传感器收集的数据进行大量处理。此外，使用硬件构造将顺序构造转换为功能等效的加速器的机制将导致高度平行，有效的传感单元，这些传感器可以更有效地执行特定的特定任务。