Collaborative Research: CNS Core: Small: From Capture to Consumption: System Challenges in Pervasive 360-Degree Video Sharing

合作研究：CNS 核心：小型：从捕获到消费：普遍 360 度视频共享的系统挑战

• 批准号: 2007176
• 负责人: Yao Liu
• 金额: $ 25万
• 依托单位: SUNY at Binghamton
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-10-01 至 2021-12-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2007176&HistoricalAwards=false
• 关键词: Collaborative; Research; CNS; Core; Small

360-degree video allows users to explore a recorded scene at any angle from the camera position. This greater flexibility provided by the 360-degree video format has led to its increased popularity. However, widespread adoption has slowed because of the large amounts of resources required by the format. This work proposes methods to improve efficiency at each stage of the 360-degree video sharing path, from video capture at the sender side to consumption at the receiver side. The project addresses these challenges via four research thrusts. The first thrust explores approaches for generating high-quality frames during real-time stitching of multiple videos captured using commodity lenses. The proposed approach applies gradient decent to refine an initial rough stitching. The second thrust investigates mechanisms to spatially adapt content to be uploaded based on the available bandwidth. The third thrust aims to optimize the fine-grained representation of omnidirectional content to achieve high projection efficiency and view efficiency. The last thrust uses edge computing techniques to optimize the look-ahead window size on the receiver side to allow bandwidth-efficient content downloading.The techniques developed during the course of the project will allow users to enjoy more efficient 360-degree video streaming applications. These techniques are not only applicable to 360-degree video streaming but will also have applications in future generations of virtual reality (VR) technologies. The project will also motivate general student interest in computer science research, directly train students during the course of the project’s research, and contribute to curriculum development. The project will produce publications, code, data, and other research artifacts. All such artifacts will be made available publicly through the URLs http://www.cs.gmu.edu/~sqchen/ and http://www.cs.binghamton.edu/~yaoliu/ during the course of the project and remain available for at least five years after completion of the project. The code will be open sourced and will also be made available at code hosting website, e.g., GitHub.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

360 度视频允许用户从摄像机位置以任何角度探索录制的场景，360 度视频格式提供的更大灵活性使其越来越受欢迎，但由于资源庞大，广泛采用的速度已经放缓。这项工作提出了提高 360 度视频共享路径的每个阶段（从发送方的视频捕获到接收方的消费）效率的方法。该项目通过四个研究重点解决了这些挑战。推力探索方法用于在使用商品镜头捕获的多个视频的实时拼接过程中生成高质量的帧。所提出的方法应用梯度下降来细化初始粗拼接。第二个重点研究了根据可用带宽在空间上调整要上传的内容的机制。第三个目标是优化全向内容的细粒度表示，以实现高投影效率和观看效率。最后一个目标使用边缘计算技术来优化接收端的前视窗口大小，以实现带宽高效的内容下载。在此过程中开发的技术项目将让用户享受更多的360度视频流应用，这些技术不仅适用于360度视频流，还将在未来几代高效的虚拟现实（VR）技术中得到应用。该项目还将激励广大学生。对计算机科学研究的兴趣，在项目研究过程中直接培训学生，并为课程开发做出贡献。该项目将产生出版物、代码、数据和其他研究成果，所有此类工件都将通过 URL 公开提供。 http://www.cs.gmu.edu/~sqchen/ 和 http://www.cs.binghamton.edu/~yaoliu/ 在项目进行期间，并在项目完成后至少五年内保持可用该代码将是开源的，也将在代码托管网站（例如 GitHub）上提供。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。