Collaborative Research: NeTS: Small: A Privacy-Aware Human-Centered QoE Assessment Framework for Immersive Videos

协作研究：NetS：小型：一种具有隐私意识、以人为本的沉浸式视频 QoE 评估框架

• 批准号: 2343618
• 负责人: Ming Li
• 金额: $ 35万
• 依托单位: University of Texas at Arlington
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2024
• 资助国家: 美国
• 起止时间: 2024-05-01 至 2027-04-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2343618&HistoricalAwards=false
• 关键词: Collaborative; Research; NeTS; Small; Privacy

Immersive videos, also known as 360-degree videos, provide viewers with a complete visual perspective of their environment. The global market for immersive video is projected to reach 22 billion U.S. dollars by 2025. With the rising popularity of 360-degree videos, network operators and service providers are increasingly keen on gaining insights into users’ Quality of Experience (QoE) perception. However, the conventional QoE models designed for two dimensional videos fall short of capturing users’ unique subjective feelings in viewing this new type of video. To bridge this gap, the project focuses on constructing human-centered models that utilize “human factors” via a variety of virtual reality (VR) onboard sensors. Leveraging multi-modal sensory readings, the researchers will tap into a direct representation of users’ perceptual experiences. This approach captures subjective feelings often missed by the objective system parameters commonly used in existing QoE models. The success of this project will provide an in-depth understanding of users’ nuanced perceptual experience while engaging with 360-degree videos. It will create opportunities for stakeholders to implement user-specific network resource optimization and video streaming strategies, ultimately enhancing customer satisfaction on an individual basis. The research outcomes will significantly contribute to the human-centered sensing and networking research communities and benefit numerous VR applications beyond 360-degree videos.This project involves three closely related research thrusts. The first thrust investigates how to extract salient features from multi-modal sensory readings for effective QoE assessment. To combat bias arising from datasets with unevenly distributed labels, novel approaches will be developed to augment underrepresented data by exploring useful information across sensing modalities. The second thrust seeks to optimize system resource utilization for QoE assessment. Multiple personalized models will share common neural network layers at edge servers for resource-efficient model hosting. Adaptive sampling will be applied at VR terminals during sensory data acquisition to maintain data utility. The third thrust focuses on devising data privacy protection mechanisms without sacrificing QoE assessment accuracy. Under the differential privacy framework, the approach features formal quantification of data correlation of multi-modality sensory data, which has been largely overlooked by prior work. The proposed research is inter-disciplinary, which spans data-driven modeling, sensing, and privacy-preserving computing. The proposed mechanisms and designs will be thoroughly evaluated via a combination of measurement campaigns, simulations, and experimental studies.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

沉浸式视频，也称为360度视频，为观众提供了对环境的完整视觉透视。到2025年，全球沉浸式视频市场预计将达到220亿美元。随着360度视频，网络运营商和服务提供商的流行越来越多，越来越热衷于对用户的经验质量（QOE）感知获得见解。但是，专为二维视频设计的常规QoE模型在观看这种新型视频时捕捉到用户独特的主观感觉的缺乏。为了弥合这一差距，该项目着重于通过板载传感器的各种虚拟现实（VR）来构建以人为因素的构建模型。利用多模式的感觉读数，研究人员将利用用户感知体验的直接表示。这种方法捕获了现有QoE模型中常用的客观系统参数通常会遇到的主观感觉。该项目的成功将在参与360度视频时对用户细致的感知体验有深入的了解。它将为利益相关者创造机会，以实施特定于用户的网络资源优化和视频流策略，最终在个人基础上提高客户满意度。研究成果将为以人为本的感官和网络研究社区做出重大贡献，并使360度视频以外的许多VR应用受益。该项目涉及三个密切相关的研究推力。第一个推力研究了如何从多模式感觉读数中提取显着特征，以进行有效的QOE评估。为了打击具有带有不均匀标签的数据集引起的偏见，将开发新的方法来通过探索跨感官方式的有用信息来增强代表性不足的数据。第二个推力旨在优化系统资源利用QOE评估。多个个性化模型将在Edge服务器上共享常见的神经网络层，以进行资源有效的模型托管。在感觉数据采集期间，将在VR终端应用自适应采样，以维持数据实用程序。第三个推力重点是设计数据隐私保护机制，而无需牺牲QoE评估精度。在差异隐私框架下，该方法具有多模式感官数据数据相关性的正式量化，这很大程度上被先前的工作忽略了。拟议的研究是跨学科的，它涵盖了数据驱动的建模，灵敏度和隐私计算。拟议的机制和设计将通过测量活动，模拟和实验研究的结合进行彻底评估。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子和更广泛影响的评估评估的评估值得支持的。