Collaborative Research: CNS Core: Medium: Foundations and Scalable Algorithms for Personalized and Collaborative Virtual Reality Over Wireless Networks

协作研究：CNS 核心：中：无线网络上个性化和协作虚拟现实的基础和可扩展算法

• 批准号: 2106090
• 负责人: Feng Qian
• 金额: $ 26.6万
• 依托单位: University of Minnesota-Twin Cities
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-10-01 至 2025-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2106090&HistoricalAwards=false
• 关键词: Collaborative; Research; CNS; Core; Medium

Virtual reality (VR) over wireless networks can provide an interactive and immersive experience for multiple users simultaneously and thus has many applications, especially in VR-based education/training. However, satisfactory personalized user experience in such wireless immersive services demands stringent performance requirements, including: (1) high-speed and high-resolution panoramic image rendering; (2) extremely low delay guarantees; and (3) seamless user experience. Besides the aforementioned requirements, collaborative user experience requires both scalability and fairness of VR service. Existing VR systems heavily rely on various heuristic designs and do not efficiently exploit VR content commonality and its predictability, which impede their large-scale deployment. This project aims to develop the theoretical foundations and complete implementation of a system for providing both personalized and scalable collaborative VR experience over wireless networks. This project will integrate machine learning, wireless networking, and mobile computing to enable high-quality and scalable wireless immersive applications on commodity mobile devices. The theory and practical implementations to be developed in this project will be integrated into both undergraduate and graduate curriculum, as well as exposing K-12 students to state-of-the-art wireless and VR technologies.The proposed designs are motivated by a number of insights that we have developed from our preliminary work, including (1) viewport-adaptive rendering; (2) commonality among VR content for multiple users to enable multicasting; and (3) predictability of VR content to enable prefetching. The proposed research will contribute to and advance both theoretical and system-oriented research in the fields of wireless networks and virtual reality. The project explicitly exploits the unique characteristics of both immersive VR applications and wireless networks, and propose the following four interdependent research thrusts: (I) Dealing with network and prediction uncertainties: This thrust will investigate algorithm designs to optimize personalized user experience given both network and viewport prediction uncertainties. (II) Meeting stringent immersive service requirements: This thrust will develop wireless scheduling algorithms that provide stringent immersive, personalized service guarantees for multiple VR users. (III) Supporting smooth collaborative interaction: This thrust will focus on the algorithm design that leverages the VR content similarities and predictabilities that naturally emerge during collaborative interactions. (IV) Scalable system integration, implementation, evaluation, and deployment: This thrust will integrate Research Thrusts I through III into a holistic system, perform system-level optimizations, and evaluate it through lab experiments and real-world classroom deployment.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

通过无线网络的虚拟现实（VR）可以同时为多个用户提供交互式和沉浸式体验，因此具有许多应用，特别是在基于VR的教育/培训中。然而，在这种无线沉浸式服务中，令人满意的个性化用户体验需要严格的性能要求，包括：（1）高速、高分辨率的全景图像渲染； (2) 极低的延迟保证； (3) 无缝的用户体验。除了上述要求外，协作用户体验还需要VR服务的可扩展性和公平性。现有的 VR 系统严重依赖各种启发式设计，无法有效利用 VR 内容的共性及其可预测性，这阻碍了其大规模部署。该项目旨在开发一个系统的理论基础和完整实现，以通过无线网络提供个性化和可扩展的协作 VR 体验。该项目将集成机器学习、无线网络和移动计算，以在商用移动设备上实现高质量和可扩展的无线沉浸式应用程序。该项目中开发的理论和实际实施将融入本科生和研究生课程，并使 K-12 学生接触最先进的无线和 VR 技术。拟议的设计受到以下因素的推动：我们从前期工作中得出的见解，包括（1）视口自适应渲染； (2) VR内容之间的通用性，供多个用户启用组播； (3) VR 内容的可预测性以实现预取。拟议的研究将有助于并推进无线网络和虚拟现实领域的理论和系统研究。该项目明确利用了沉浸式 VR 应用和无线网络的独特特征，并提出了以下四个相互依赖的研究主旨： (I) 处理网络和预测不确定性：该主旨将研究算法设计，以在给定网络和无线网络的情况下优化个性化用户体验。视口预测的不确定性。 （二）满足严格的沉浸式服务要求：该重点将开发无线调度算法，为多个VR用户提供严格的沉浸式、个性化服务保证。 (III) 支持流畅的协作交互：该重点将侧重于利用协作交互过程中自然出现的 VR 内容相似性和可预测性的算法设计。 （IV）可扩展的系统集成、实施、评估和部署：该主旨将把研究主旨 I 至 III 集成到一个整体系统中，进行系统级优化，并通过实验室实验和实际课堂部署对其进行评估。该奖项反映了通过使用基金会的智力价值和更广泛的影响审查标准进行评估，NSF 的法定使命被认为值得支持。

项目成果

How Will VR Enter University Classrooms? Multi-stakeholders Investigation of VR in Higher Education

VR将如何走进大学课堂？

• DOI: 10.1145/3491102.3517542
• 发表时间: 2022-04
• 期刊: ACM CHI 2022
• 作者: Jin, Qiao;Liu, Yu;Yarosh, Svetlana;Han, Bo;Qian, Feng
• 通讯作者: Qian, Feng

Enhancing Quality of Experience for Collaborative Virtual Reality with Commodity Mobile Devices

使用商用移动设备提高协作虚拟现实的体验质量

• DOI: 10.1109/icdcs54860.2022.00102
• 发表时间: 2022-07-01
• 期刊: 2022 IEEE 42nd International Conference on Distributed Computing Systems (ICDCS)
• 作者: Jiangong Chen;Feng Qian;Bin Li
• 通讯作者: Bin Li

Collaborative Online Learning with VR Video: Roles of Collaborative Tools and Shared Video Control

VR 视频协作在线学习：协作工具和共享视频控制的作用

• DOI: 10.1145/3544548.3581395
• 发表时间: 2023-04-19
• 期刊: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems
• 作者: Qiao Jin;Yu Liu;Ruixuan Sun;Chen Chen;Puqi Zhou;Bo Han;Feng Qian;S. Yarosh
• 通讯作者: S. Yarosh

Vues: Practical Mobile Volumetric Video Streaming Through Multiview Transcoding

Vues：通过多视图转码实现实用的移动体积视频流

• DOI: 10.1145/3495243.3517027
• 发表时间: 2022-10
• 期刊: ACM MobiCom 2022
• 作者: Liu, Yu;Han, Bo;Qian, Feng;Narayanan, Arvind;Zhang, Zhi
• 通讯作者: Zhang, Zhi