Collaborative Research: CNS Core: Medium: Panoptes: Next Generation Multi-Perspective Video Delivery at Internet Scale

合作研究：CNS 核心：媒介：Panoptes：互联网规模的下一代多视角视频传输

• 批准号: 1956018
• 负责人: Sanjay Rao
• 金额: $ 60万
• 依托单位: Purdue University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1956018&HistoricalAwards=false
• 关键词: Collaborative; Research; CNS; Core; Medium

The success of streaming video has generated interest in newer forms of multi-perspective video content, such as those generated by 360-degree cameras, multi-angle camera arrays, or light-field cameras. The immersive experience provided by these cameras can enhance user satisfaction in domains including sports, training (e.g., construction safety), and virtual exploration (e.g., college walkthroughs, historical sites). With content from these cameras, users do not just passively consume content, but may interactively traverse the content along many different paths from a perspective of their choice, with different users observing different perspectives of the same content. To support this at Internet-scale is challenging; client players must be able to switch perspectives with low latency, the perspectives may need to be generated on demand by video servers, and the infrastructure must support a variety of devices for capture and consumption of this content.This project explores architectural enhancements, algorithms, and techniques to deliver multi-perspective video at Internet-scale. It couples delivery optimization with video coding and human computer interaction. The project will develop interactivity abstractions by which content publishers can specify the range of perspectives users are permitted to choose from at each point in the video. The interactivity specified in a video will (i) drive perspective coding and novel dynamic perspective generation algorithms; (ii) enable infrastructure provisioning to meet Content Delivery Network (CDN) storage and cost constraints; and (iii) will guide adaptive perspective retrieval from CDN servers or from nearby caches. Finally, the project will explore methods to predict and guide user behavior to improve delivery quality based on user studies. By enabling more responsive and predictable user experience for multi-perspective video at Internet scale, and by designing networks that can cope with growth in such video, the project will impact the media, IT, education, and networking industries. It will convene an Industrial Advisory Board comprising relevant practitioners and organize workshops to foster collaborations. The research will not only extensively involve graduate students, but also undergraduate students from underrepresented communities, will create new curriculum material, as well as a new undergraduate design project that involves students ranging from the freshman to senior year with strong participation from women and underrepresented students. The results from the project will be available at: https://engineering.purdue.edu/~isl/multi-perspective-video.htmland https://nsl.usc.edu/projects/multi-perspective-video/The results will be available throughout the duration of the project, and for three years after the project ends.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

流视频的成功引起了人们对较新形式的多人视频内容的兴趣，例如由360度摄像机，多角度相机阵列或光场摄像机生成的内容。这些相机提供的沉浸式体验可以提高体育，培训（例如建筑安全）和虚拟探索等领域的用户满意度（例如，大学演练，历史遗迹）。借助这些相机的内容，用户不仅可以被动地消费内容，而且可以从选择的角度从许多不同的路径上进行交互式遍历内容，而不同的用户可以观察到同一内容的不同观点。在互联网规模上支持这一点是具有挑战性的。客户玩家必须能够以较低的延迟进行切换，视频服务器可能需要按需生成视角，并且基础架构必须支持各种设备以捕获和消耗此内容。该项目探讨了架构增强功能，算法和技术以在Internet-Scale中提供多人视频。它将交付优化与视频编码和人类计算机交互。该项目将开发互动抽象，内容发布者可以通过这些抽象来指定视频中每个点的观点范围。视频中指定的交互性（i）驱动透视编码和新颖的动态透视生成算法； （ii）启用基础架构提供以满足内容交付网络（CDN）存储和成本限制； （iii）将指导从CDN服务器或附近缓存的自适应透视图检索。最后，该项目将探索预测和指导用户行为以根据用户研究提高交付质量的方法。通过在Internet范围内为多观点视频提供更响应迅速和可预测的用户体验，并设计可以应对此类视频增长的网络，该项目将影响媒体，IT，教育和网络行业。它将召集一个由相关从业者组成的工业顾问委员会，并组织研讨会以促进合作。这项研究不仅将广泛涉及研究生，而且还将与来自代表性不足的社区的本科生有关，还将创建新的课程材料，以及一个新的大学生设计项目，涉及从新生到高年级的学生，妇女和人为不足的学生的参与。该项目的结果将在以下网址提供：https：//engineering.purdue.edu/~isl/multi-pective-video.htmland https：//nsl.usc.edu/projects/projects/multi-ppective-video/ the Reading the Project and The Project in The Project and the Project and to and to and to and。使用基金会的知识分子优点和更广泛的审查标准，通过评估被认为值得支持。