Collaborative Research: CNS Core: HCC: Small: Enabling Efficient Computer Systems for Augmented and Virtual Reality: A Perception-Guided Approach

合作研究：CNS 核心：HCC：小型：为增强现实和虚拟现实启用高效计算机系统：感知引导方法

• 批准号: 2225860
• 负责人: Yuhao Zhu
• 金额: $ 29.99万
• 依托单位: University of Rochester
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-11-01 至 2025-10-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2225860&HistoricalAwards=false
• 关键词: Collaborative; Research; CNS; Core; HCC

This research project designs software-hardware collaborative mechanisms to bridge the gap between human visual perception and augmented/virtual reality (AR/VR) systems design and ultimately deliver an order of magnitude efficiency gains. Achieving this goal requires first developing computational models to quantitatively capture human visual perception, and then investigating new techniques across the systems stack that leverage the perceptual models for efficient processing. The intellectual merit of the project is the pursuit of three classes of techniques pertaining to exploiting unique characteristics of human perception to build efficient AR/VR systems: 1) identify key aspects of visual stimuli that simultaneously influence viewing experience and system efficiency and investigate lightweight modeling strategies through extensive psychophysical experiments, 2) investigate enabling techniques across the entire application pipeline (from algorithms and systems to the display) that leverage the perceptual models for improving system efficiency, and 3) develop a data-driven, learning-based approach to tune the entire system while satisfying perceptual and efficiency requirements.The research agenda is complemented by an educational/outreach agenda. The PIs will (1) offer summer research and programming opportunities to high school students in Rochester and New York, (2) introduce new courses/modules in both University of Rochester and NYU on visual perception and its connection to computer systems design, (3) reach out and collaboration with industry partners to explore technology transfer possibilities, and (4) offer undergraduate students inclusive opportunities for hands-on experience in emerging application domains and hardware acceleration. Broadly, the integrated research and education activities will lead the way to an explosion of new products and ideas, which in turn addresses key technological and societal issues in areas such as healthcare, education, remote communication and interaction, professional training, and industrial design.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该研究项目设计软件-硬件协作机制，以弥合人类视觉感知和增强/虚拟现实 (AR/VR) 系统设计之间的差距，并最终实现一个数量级的效率提升。实现这一目标需要首先开发计算模型来定量捕获人类视觉感知，然后研究跨系统堆栈的新技术，利用感知模型进行高效处理。该项目的智力优点是追求三类技术，这些技术涉及利用人类感知的独特特征来构建高效的 AR/VR 系统：1）识别同时影响观看体验和系统效率的视觉刺激的关键方面，并研究轻量级建模通过广泛的心理物理学实验来制定策略，2）研究整个应用程序管道（从算法和系统到显示）的支持技术，利用感知模型来提高系统效率，3）开发一种数据驱动、基于学习的方法来调整整个系统同时满足感知和效率要求。研究议程由教育/推广议程补充。 PI 将 (1) 为罗彻斯特和纽约的高中生提供暑期研究和编程机会，(2) 在罗彻斯特大学和纽约大学引入有关视觉感知及其与计算机系统设计的联系的新课程/模块，(3 ) 与行业合作伙伴进行接触和合作，探索技术转让的可能性，(4) 为本科生提供在新兴应用领域和硬件加速方面获得实践经验的包容性机会。从广义上讲，综合研究和教育活动将导致新产品和创意的爆炸式增长，从而解决医疗保健、教育、远程通信和互动、专业培训和工业设计等领域的关键技术和社会问题。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Color-Perception-Guided Display Power Reduction for Virtual Reality

• DOI: 10.1145/3550454.3555473
• 发表时间: 2022-09
• 期刊: ACM Transactions on Graphics (TOG)
• 作者: Budmonde Duinkharjav;Kenneth Chen;Abhishek Tyagi;Jiaying He;Yuhao Zhu;Qiuyue Sun