ICE-T: RC: Millimeter Wave Communications and Edge Computing for Next Generation Tetherless Mobile Virtual Reality

ICE-T：RC：下一代无线移动虚拟现实的毫米波通信和边缘计算

• 批准号: 2032033
• 负责人: Jacob Chakareski
• 金额: $ 27.97万
• 依托单位: New Jersey Institute of Technology
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-04-20 至 2024-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2032033&HistoricalAwards=false
• 关键词: ICE; RC; Millimeter; Wave; Communications

Virtual and augmented reality (VR/AR) technologies hold tremendous potential to advance our society, having impact on quality of life, environmental and energy conservation, and the world economy. However, two main challenges stand in the way of realizing this vision. These applications are hyper-data-intensive and require ultra-low latency, neither of which can be met by current and upcoming conventional networking methods and systems. These presently limit VR/AR applications to an offline operation, synthetic content, high-end wired equipment, and gaming/entertainment settings. This project envision a novel system at the intersection of millimeter-wave communication (mmWave) and edge computing that aims to overcome these challenges to bring us closer to the next generation tetherless VR/AR societal applications. The project will make notable contributions to the emerging area of networked VR/AR application systems and communications, leading to advances in numerous socially relevant applications, e.g., search and rescue, and disaster response. It will also facilitate fundamental research in the general application area of high-volume high-speed/low-latency data transfer in emerging settings. Beyond the direct scientific and technology impacts and their broader effects on society, educational, outreach, international collaboration, and scientific leadership activities will be pursued as an integral part of the project.Overcoming the broad performance gap between present and upcoming networked systems capabilities and anticipated requirements of next generation applications will require novel holistic approaches to capture, coding, networking, and reconstruction/navigation of VR/AR data. Towards this objective, the project will investigate a futuristic 5G heterogeneous cellular network system that integrates radio frequency (RF) and millimeter wave communication, and viewport-adaptive space-time scalable VR signal tiling, for multi-path streaming of 360-degree tetherless mobile VR applications. In this setting, the project will pursue the following synergistic investigations: (1) Navigation-aware scalable VR signal tiling to enable interactive streaming of only the data truly needed by the user during navigation; (2) Deep machine learning for user navigation prediction to assist the envisioned resource allocation methods. (3) Space-time scalable rateless code construction for effective source-channel VR signal representation to protect against prospective transmission errors. (4) Dynamic rate-distortion optimized strategies for hybrid RF-mmWave multi-path VR streaming and analysis of the foundations of the interdependencies between the VR signal tiling design and the characteristics of the two network paths. (5) Analysis of the fundamental trade-offs between edge computing and communication that arise here and pursuit of optimization methods that will leverage them to maximize the system efficiency. (6) Graph-theoretic analysis of the problem of dynamic mmWave transmitter to VR user assignment. (7) Network slicing for parallel operation with other applications. Extensive integration and experimentation will be carried out to assess, validate, and prototype the enabled research advances in practical settings.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

虚拟和增强现实（VR/AR）技术具有巨大的潜力，可以影响我们的社会，对生活质量，环境和节能以及世界经济产生影响。但是，两个主要的挑战阻碍了实现这一愿景。这些应用程序非常密集，需要超低潜伏期，这两个延迟都无法通过当前和即将到来的常规网络方法和系统来满足。这些目前将VR/AR应用程序限制在离线操作，合成内容，高端有线设备和游戏/娱乐设置中。该项目设想在毫米波通信（MMWave）和边缘计算的交汇处，旨在克服这些挑战，使我们更接近下一代无限VR/AR社会应用。该项目将为网络VR/AR应用程序系统和通信的新兴领域做出显着贡献，从而在众多与社会相关的应用程序（例如搜索和救援）以及灾难响应中取得进步。它还将促进新兴环境中高体积/低延迟数据传输的一般应用领域的基础研究。 Beyond the direct scientific and technology impacts and their broader effects on society, educational, outreach, international collaboration, and scientific leadership activities will be pursued as an integral part of the project.Overcoming the broad performance gap between present and upcoming networked systems capabilities and anticipated requirements of next generation applications will require novel holistic approaches to capture, coding, networking, and reconstruction/navigation of VR/AR data.为了实现这一目标，该项目将研究一个未来派的5G异质蜂窝网络系统，该系统集成了射频（RF）和毫米波通信，以及视图自动时空可伸缩VR信号瓷砖，用于360度无线无线无线的多路径流动。在这种情况下，该项目将进行以下协同调查：（1）导航可扩展的VR信号瓷砖，以启用仅在导航期间用户真正需要的数据的交互式流； （2）用户导航预测的深度机器学习，以帮助设想的资源分配方法。 （3）有效源通道VR信号表示的时空可扩展无权代码构建，以防止前瞻性传输错误。 （4）混合RF-MWAVE多路径VR流的动态率优化策略以及对VR信号瓷砖设计与两个网络路径特征之间相互依赖基础的基础的分析。 （5）分析此处出现的边缘计算与通信之间的基本权衡以及追求优化方法，以利用它们来最大程度地提高系统效率。 （6）动态MMWave发射器到VR用户分配问题的图理论分析。 （7）与其他应用程序并行操作的网络切片。将进行广泛的整合和实验，以评估，验证和原型实践环境中实现的研究进展。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子和更广泛影响的审查标准的评估来获得支持的。

项目成果

UAV-IoT for Next Generation Virtual Reality

• DOI: 10.1109/tip.2019.2921869
• 发表时间: 2019-12-01
• 期刊: IEEE TRANSACTIONS ON IMAGE PROCESSING
• 影响因子: 10.6
• 作者: Chakareski, Jacob

FBDT: Forward and Backward Data Transmission Across RATs for High Quality Mobile 360-Degree Video VR Streaming

• DOI: 10.1145/3587819.3590987
• 发表时间: 2023-06
• 期刊: Proceedings of the 14th Conference on ACM Multimedia Systems
• 作者: S. Srinivasan;Samuel Shippey;Ehsan Aryafar;Jacob Chakareski

Lifetime Maximization in Mobile Edge Computing Networks

• DOI: 10.1109/tvt.2020.2965440
• 发表时间: 2020-01
• 期刊: IEEE Transactions on Vehicular Technology
• 影响因子: 6.8
• 作者: Sabyasachi Gupta;Jacob Chakareski

Geometric Programming for Lifetime Maximization in Mobile Edge Computing Networks

• DOI: 10.1109/globecom38437.2019.9013487
• 发表时间: 2019-12
• 期刊: 2019 IEEE Global Communications Conference (GLOBECOM)
• 作者: Sabyasachi Gupta;Jacob Chakareski

mmWave Networking and Edge Computing for Scalable 360° Video Multi-User Virtual Reality

• DOI: 10.1109/tip.2022.3228521
• 发表时间: 2022-12
• 期刊: IEEE Transactions on Image Processing
• 影响因子: 10.6
• 作者: Sabyasachi Gupta;Jacob Chakareski;P. Popovski