SHF: Medium: A Visual Cloud for Virtual Reality Applications

SHF：Medium：虚拟现实应用程序的视觉云

• 批准号: 1703051
• 负责人: Magdalena Balazinska
• 金额: $ 91.6万
• 依托单位: University of Washington
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-08-01 至 2022-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1703051&HistoricalAwards=false
• 关键词: SHF; Medium; Visual; Cloud; Virtual

The ability to collect videos can revolutionize interactions with the world by enabling powerful virtual reality video applications in education, tourism, tele-presence and others. These new applications involve processing and serving 360-degree stereoscopic videos, which require a dramatic improvement in technology to manage and process the massive-scale visual data necessary for truly immersive experiences. Systems that support VR video also represent an excellent educational tool as students can experience scenes in a truly immersive way and hence better convey content. This project builds a visual cloud that provides seamless access to a new database management system with hardware acceleration and edge computation that enables the efficient and real-time management of massive-scale image data and virtual reality (VR) applications built on top of it. The project develops a new hardware and software stack for VR data processing with execution in public clouds. The stack includes a new storage manager that significantly increases data ingest and retrieval throughputs for multidimensional array data compared with existing systems, as motivated by the extreme needs of VR applications. The storage manager utilizes novel hardware technologies (non-volatile memory) and provides novel approximate and multi-resolution data storage capabilities. The project also develops a new runtime system for high-throughput and large-scale array processing by developing a new API for expressing VR pipelines as a graph of user-defined functions, a library of specialized implementations of known VR algorithms for different types of hardware (CPU, GPU, FPGA, and 3D XPoint), and associated optimizers and schedulers. Finally, the project develops new techniques to enable real-time VR applications. They include an FPGA-based acceleration platform for real-time VR video processing and algorithms and software components for prefetching and caching VR data close to the viewers and processing that data in the viewing device. http://visualcloud.cs.washington.edu

收集视频的能力可以通过在教育，旅游，电视节目等中实现强大的虚拟现实视频应用程序来彻底改变与世界的互动。这些新的应用程序涉及处理和提供360度立体视频，这些视频需要对技术进行巨大改进，以管理和处理真正沉浸式体验所需的大规模视觉数据。支持VR视频的系统也代表了一种出色的教育工具，因为学生可以真正身临其境的方式体验场景，从而更好地传达内容。该项目构建了一个视觉云，可通过硬件加速度和边缘计算无缝访问新的数据库管理系统，从而实现了在其顶部构建的大规模尺度图像数据和虚拟现实（VR）应用程序（VR）应用程序的高效和实时管理。该项目开发了一个新的硬件和软件堆栈，用于通过公共云中的执行来处理VR数据处理。该堆栈包括一个新的存储管理器，与现有系统相比，与现有系统相比，多维阵列数据的数据摄入和检索吞吐量，这是由VR应用程序的极端需求所激发的。存储经理利用新颖的硬件技术（非挥发性内存），并提供新颖的近似和多分辨率数据存储功能。 The project also develops a new runtime system for high-throughput and large-scale array processing by developing a new API for expressing VR pipelines as a graph of user-defined functions, a library of specialized implementations of known VR algorithms for different types of hardware (CPU, GPU, FPGA, and 3D XPoint), and associated optimizers and schedulers.最后，该项目开发了新技术来启用实时VR应用程序。它们包括一个基于FPGA的加速平台，用于实时VR视频处理以及算法和软件组件，用于预取料和缓存VR数据靠近观众，并在查看设备中处理该数据。 http://visualcloud.cs.washington.edu

项目成果

VSS: A Storage System for Video Analytics

• DOI: 10.1145/3448016.3459242
• 发表时间: 2021-03
• 期刊: Proceedings of the 2021 International Conference on Management of Data
• 作者: Brandon Haynes;Dong He

Perceptual Compression for Video Storage and Processing Systems

• DOI: 10.1145/3357223.3362725
• 发表时间: 2019-02
• 期刊: Proceedings of the ACM Symposium on Cloud Computing
• 作者: Amrita Mazumdar;Brandon Haynes;M. Balazinska;L. Ceze;Alvin Cheung;M. Oskin

VOCAL: Video Organization and Interactive Compositional AnaLytics

• 发表时间: 2022
• 作者: Maureen Daum;Enhao Zhang;Dong He;M. Balazinska;Brandon Haynes;Ranjay Krishna;Apryle Craig;Aaron J. Wirsing

LightDB: A DBMS for Virtual Reality Video

• DOI: 10.14778/3231751.3231768
• 发表时间: 2018-06
• 期刊: Proc. VLDB Endow.
• 作者: Brandon Haynes;Amrita Mazumdar;Armin Alaghi;M. Balazinska;L. Ceze;Alvin Cheung

VisualWorldDB: A DBMS for the Visual World. Brandon Haynes, Maureen Daum, Amrita Mazumdar, Magdalena Balazinska, Alvin Cheung, and Luis Ceze. CIDR, 2020.

VisualWorldDB：视觉世界的 DBMS。

• 发表时间: 2020
• 期刊: Conference on Innovative Data Systems Research (CIDR
• 作者: Brandon Haynes, Maureen Daum