Low-delay reliable video delivery for real-time immersive applications

适用于实时沉浸式应用的低延迟可靠视频传输

• 批准号: RGPIN-2022-02995
• 负责人: Coulombe, Stéphane
• 金额: $ 3.35万
• 依托单位: École de technologie supérieure
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=754229
• 关键词: Low; delay; reliable; video; delivery

Recent years have witnessed a rapid deployment of video devices, applications and systems, and this growth will only intensify. Large amounts of video data will be streamed in very innovative applications such as live immersive video (e.g., 360-degree video), visual remote machinery control, remote surgery and live inter-vehicular communications. A wide diversity of interactive video applications are deployed on heterogeneous networks, and ensuring the reliable delivery of video data, especially under low delay constraints, is highly challenging. Specifically, transmission errors and packet losses over wireless networks cause significant delays and visual quality degradation. Thus, in seeking to provide the best quality of experience to end users, the ability to transport video content in a fast and reliable manner constitutes one of the most crucial challenges for modern networks, such as 5G. This research program aims mainly to propose novel solutions to the problem of ensuring the reliable delivery of video data over heterogeneous networks, under very low delay constraints, in the context of immersive video applications. Specifically, the applicant and his team will create promising methods to protect video data against losses and transmission errors, in addition to methods to correct and repair video content. These will remove the need for retransmissions, which are undesirable or unsuitable for very low-delay applications. In this quest, they will pursue five research tracks. First, they will create new classes of forward error correction (FEC) methods suitable for low-delay interactive video applications. Second, they will propose a radically new approach combining their newly developed algorithms for correcting erroneous video packets, using the Cyclic Redundancy Check information, with these new FEC classes, to protect the system against both bit errors and packets losses. Third, they will develop new machine learning approaches to dynamically adapt the FEC parameters to the video content and ever-changing network conditions in order to ensure error-free video reconstruction while meeting delay constraints. Fourth, they will apply these technologies to the coding and delivery of tiles-based 360-degree video by exploiting coding tools found in the new versatile video coding (VVC)/H.266 standard. Finally, they will propose new deep-learning approaches to evaluate the quality of damaged and repaired videos. The advances provided by this research will significantly improve the quality and usability of interactive and immersive video services, as well as other delay-sensitive applications, and enable the emergence of unprecedented media experiences. As for their previous research, Canada will benefit strongly from the intellectual property and commercialization of the technologies therein and from the training of highly qualified personnel in the fast-growing field of immersive video.

近年来，视频设备，应用程序和系统迅速部署，这种增长只会加剧。大量视频数据将在非常创新的应用程序中流式传输，例如实时的身临其境视频（例如360度视频），可视远程机械控制，远程手术和实时车间间通信。各种交互式视频应用程序都部署在异质网络上，并确保可靠的视频数据传递，尤其是在低延迟约束下，这是极具挑战性的。具体而言，无线网络上的传输错误和数据包损失会导致重大延迟和视觉质量降解。因此，在寻求为最终用户提供最佳体验质量时，以快速可靠的方式运输视频内容的能力构成了现代网络（例如5G）最关键的挑战之一。该研究计划的目的主要旨在提出新的解决方案，以确保在沉浸式视频应用程序的背景下，在非常低的延迟约束下，在非常低的延迟限制下可靠地传递视频数据。具体而言，除了纠正和维修视频内容的方法外，申请人及其团队还将创建有前途的方法来保护视频数据免受损失和传输错误的影响。这些将消除重新启动的需求，这是不受欢迎的或不适合非常低延迟的应用程序的。在此任务中，他们将追求五个研究轨道。首先，他们将创建适用于低延迟交互式视频应用程序的新型远期错误校正（FEC）方法。其次，他们将提出一种新的新方法，结合了新开发的算法，用于使用环状冗余检查信息以及这些新的FEC类，以纠正错误的视频数据包，以保护系统免受位错误和数据包损失。第三，他们将开发新的机器学习方法，以动态地使FEC参数适应视频内容和不断变化的网络条件，以确保在满足延迟约束时确保无错误的视频重建。第四，他们将通过利用在新的Versatile视频编码（VVC）/H.266标准中发现的编码工具来将这些技术应用于基于瓷砖的360度视频的编码和交付。最后，他们将提出新的深度学习方法，以评估受损和维修视频的质量。这项研究提供的进步将显着提高交互式和沉浸式视频服务的质量和可用性，以及其他延迟敏感的应用程序，并能够出现前所未有的媒体体验。至于他们先前的研究，加拿大将从其中的知识产权和商业化中受益匪浅，以及在快速增长的沉浸式视频领域中对高素质人员的培训。