Collaborative Research: CNS Core: HCC: Small: Enabling Efficient Computer Systems for Augmented and Virtual Reality: A Perception-Guided Approach

合作研究：CNS 核心：HCC：小型：为增强现实和虚拟现实启用高效计算机系统：感知引导方法

• 批准号: 2225860
• 负责人: Yuhao Zhu
• 金额: $ 29.99万
• 依托单位: University of Rochester
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-11-01 至 2025-10-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2225860&HistoricalAwards=false
• 关键词: Collaborative; Research; CNS; Core; HCC

This research project designs software-hardware collaborative mechanisms to bridge the gap between human visual perception and augmented/virtual reality (AR/VR) systems design and ultimately deliver an order of magnitude efficiency gains. Achieving this goal requires first developing computational models to quantitatively capture human visual perception, and then investigating new techniques across the systems stack that leverage the perceptual models for efficient processing. The intellectual merit of the project is the pursuit of three classes of techniques pertaining to exploiting unique characteristics of human perception to build efficient AR/VR systems: 1) identify key aspects of visual stimuli that simultaneously influence viewing experience and system efficiency and investigate lightweight modeling strategies through extensive psychophysical experiments, 2) investigate enabling techniques across the entire application pipeline (from algorithms and systems to the display) that leverage the perceptual models for improving system efficiency, and 3) develop a data-driven, learning-based approach to tune the entire system while satisfying perceptual and efficiency requirements.The research agenda is complemented by an educational/outreach agenda. The PIs will (1) offer summer research and programming opportunities to high school students in Rochester and New York, (2) introduce new courses/modules in both University of Rochester and NYU on visual perception and its connection to computer systems design, (3) reach out and collaboration with industry partners to explore technology transfer possibilities, and (4) offer undergraduate students inclusive opportunities for hands-on experience in emerging application domains and hardware acceleration. Broadly, the integrated research and education activities will lead the way to an explosion of new products and ideas, which in turn addresses key technological and societal issues in areas such as healthcare, education, remote communication and interaction, professional training, and industrial design.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该研究项目设计了软件硬件协作机制，以弥合人类视觉感知与增强/虚拟现实（AR/VR）系统设计之间的差距，并最终提供了效率增长的顺序。实现此目标需要首先开发计算模型，以定量捕获人类的视觉感知，然后研究整个系统堆栈中利用感知模型进行有效处理的新技术。该项目的智力优点是追求与利用人类感知的独特特征有关的三类技术，以建立有效的AR/VR系统：1）确定视觉刺激的关键方面，这些方面同时影响观看经验和系统效率，并通过对整个系统进行策略（2）调查策略（2）调查策略（2）利用提高系统效率的感知模型，以及3）开发一种基于数据驱动的，基于学习的方法来调整整个系统，同时满足感知和效率要求。研究议程得到了教育/推广议程的补充。 The PIs will (1) offer summer research and programming opportunities to high school students in Rochester and New York, (2) introduce new courses/modules in both University of Rochester and NYU on visual perception and its connection to computer systems design, (3) reach out and collaboration with industry partners to explore technology transfer possibilities, and (4) offer undergraduate students inclusive opportunities for hands-on experience in emerging application domains and hardware acceleration.广泛地说，综合研究和教育活动将导致新产品和思想的爆炸式爆炸，这又解决了诸如医疗保健，教育，远程沟通和互动，专业培训和工业设计等领域的关键技术和社会问题。该奖项反映了NSF的立法使命，并认为通过基金会的知识效果和广泛的评估，可以通过评估来进行评估。

项目成果

Color-Perception-Guided Display Power Reduction for Virtual Reality

• DOI: 10.1145/3550454.3555473
• 发表时间: 2022-09
• 期刊: ACM Transactions on Graphics (TOG)
• 作者: Budmonde Duinkharjav;Kenneth Chen;Abhishek Tyagi;Jiaying He;Yuhao Zhu;Qiuyue Sun