VEC: Medium: Large-Scale Visual Recognition: From Cloud Data Centers to Wearable Devices

VEC：中：大规模视觉识别：从云数据中心到可穿戴设备

• 批准号: 1539011
• 负责人: Thomas Wenisch
• 金额: $ 96万
• 依托单位: Regents of the University of Michigan - Ann Arbor
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2015
• 资助国家: 美国
• 起止时间: 2015-10-01 至 2021-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1539011&HistoricalAwards=false
• 关键词: VEC; Medium; Large; Scale; Visual

Advances in computer hardware and software promise to revolutionize the ways in which society interacts with visual information. However, visual recognition systems are limited by the lack of a practical means to classify the millions of concepts that arise in visual scenes and thus efficiently recognize when a small number of these concepts appear in a given scene. Furthermore, while real-time processing of visual data could significantly expand our perception of our surroundings, state-of-the-art vision systems cannot currently be implemented on wearable devices such as smartphones due to the limited heat dissipation (e.g., no fans or liquid cooling) and power such devices can provide. This research will overcome these challenges by developing artificial intelligence (AI) systems that efficiently manage the resources most crucial for high-performance wearable-based visual recognition, including the wearable device's real-time power consumption and computation. These systems will be empowered to initiate bursts of intense computation that are thermally managed by materials within the wearable device which are engineered to melt during heavy heating and solidify between bursts. Moreover, the AI systems will govern the communication between the device and external (cloud-based) computation resources as well as large-scale visual concept databases housed in data centers, thus providing extreme performance in a wearable form factor. Central concepts of this work will be integrated in undergraduate and graduate coursework, and a demonstration system will be made available to the research community and used in educational modules for high school students.This effort seeks to advance the core capabilities of large-scale visual recognition by co-designing visual models and computing infrastructure. The goal is to enable encyclopedic, real-time visual recognition through seamless integration of visual computing on wearable devices and in the cloud. The PIs envision a wearable visual recognition system that continuously captures live video input while providing intelligent, real-time assistance through automatic or on-demand visual recognition by means of a combination of computation at the device and offloading to the cloud. Such a system is not currently feasible due to a number of fundamental challenges. First, the severe energy and thermal constraints of wearable devices render them incapable of performing the intensive computation necessary for visual recognition. Second, it remains an open question how to support encyclopedic recognition in terms of both visual models and data center infrastructure. In particular, it remains unclear how current visual models, although highly successful at recognizing 1,000 object categories, can scale to millions or more distinct visual concepts. Moreover, such an encyclopedic visual model must be supported through data center infrastructure, but little progress has been made on how to build such infrastructure. This project addresses these fundamental challenges through an interdisciplinary approach integrating computer vision, hardware architecture, VLSI design, and heat transfer. The PIs will investigate three research thrusts. In Thrust 1, the PIs will develop a new type of deep neural networks that allow resource-efficient execution of modules. This new framework provide a unified way to design, learn, and run scalable visual models that can maximize the utility of recognition subject to resource constraints, such as latency, energy, or thermal dissipation of a wearable device. In Thrust 2, the PIs will design and fabricate a visual processing chip capable of computational sprinting (bursts of extreme computation well above steady-state thermal dissipation capabilities), leveraging the new framework developed in Thrust 1. In Thrust 3, the PIs will design datacenter infrastructure that supports large-scale hierarchical indexing of visual concepts for encyclopedic recognition, with a focus on latency, throughput, and energy efficiency. Finally, the PIs will build a demonstration system to evaluate the proposed algorithms, software, and hardware components and to assess the overall performance of an end-to-end system. The project web site (http://mivec.eecs.umich.edu/) will provide access to the results of this research including technical reports, datasets, and source code.

计算机硬件和软件的进步有望改变社会与视觉信息互动的方式。但是，视觉识别系统受到缺乏实际手段来对视觉场景中出现的数百万个概念进行分类的限制，从而有效地识别出少数这些概念在给定场景中。此外，尽管视觉数据的实时处理可能会大大扩展我们对周围环境的看法，但由于有限的散热耗散（例如，没有风扇或液体冷却），目前无法在智能手机等可穿戴设备（例如智能手机）上实施最先进的视觉系统，并且此类设备可以提供。这项研究将通过开发人工智能（AI）系统来克服这些挑战，这些系统有效地管理对高性能可穿戴的视觉识别最关键的资源，包括可穿戴设备的实时功耗和计算。这些系统将有权启动强烈的计算爆发，这些计算由可穿戴设备中的材料进行热管理，这些材料在重型加热期间经过精心融化，并在爆发之间固化。此外，AI系统将控制设备与外部（基于云的）计算资源之间的通信，以及数据中心中包含的大规模视觉概念数据库，从而在可穿戴的外形方面提供了极端性能。这项工作的核心概念将集成在本科和研究生课程中，并且将向研究社区提供一个示范系统，并用于高中生的教育模块中。这项工作旨在通过共同设计的视觉模型和计算基础架构来提高大规模视觉识别的核心能力。目的是通过在可穿戴设备和云中无缝集成视觉计算来实现百科全书，实时视觉识别。 PIS设想了一个可穿戴的视觉识别系统，该系统通过自动或按需视觉识别来通过设备上的计算并将其卸载到云中，通过自动或按需视觉识别提供智能，实时的帮助。由于许多基本挑战，目前不可行这样的系统。首先，可穿戴设备的严重能量和热约束使它们无法执行视觉识别所需的密集计算。其次，它仍然是一个开放的问题，如何在视觉模型和数据中心基础架构方面支持百科全书识别。特别是，尚不清楚当前的视觉模型虽然在识别1,000个对象类别方面非常成功，但可以扩展到数百万或更截然不同的视觉概念。此外，这种百科全书视觉模型必须通过数据中心基础架构支持，但是如何建立此类基础架构的进展很少。该项目通过跨学科的方法来解决这些基本挑战，从而整合了计算机视觉，硬件体系结构，VLSI设计和传热。 PI将研究三个研究推力。在推力1中，PI将开发一种新型的深神经网络，以允许资源有效执行模块。这个新框架为设计，学习和运行可扩展的视觉模型提供了一种统一的方法，该模型可以最大程度地提高识别效用，但要受资源约束（例如延迟，能量或可穿戴设备的热量消散）。 In Thrust 2, the PIs will design and fabricate a visual processing chip capable of computational sprinting (bursts of extreme computation well above steady-state thermal dissipation capabilities), leveraging the new framework developed in Thrust 1. In Thrust 3, the PIs will design datacenter infrastructure that supports large-scale hierarchical indexing of visual concepts for encyclopedic recognition, with a focus on latency, throughput,和能源效率。最后，PI将建立一个演示系统，以评估所提出的算法，软件和硬件组件，并评估端到端系统的整体性能。项目网站（http://mivec.eecs.umich.edu/）将提供对本研究结果的访问，包括技术报告，数据集和源代码。