SHF: Small: Turning Visual Noise into Hardware Efficiency: Viewer-Aware Energy-Quality Adaptive Mobile Video Storage

SHF：小：将视觉噪声转化为硬件效率：观看者感知的能源质量自适应移动视频存储

• 批准号: 1815430
• 负责人: Na Gong
• 金额: $ 30万
• 依托单位: North Dakota State University Fargo
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-10-01 至 2018-11-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1815430&HistoricalAwards=false
• 关键词: SHF; Small; Turning; Visual; Noise

Mobile devices, such as smart phones, are being increasingly utilized for watching videos, since they can be conveniently used for this purpose anywhere anytime, such as commuting on a subway or train, sitting in a waiting room, or lounging at home. Due to the large data size and intensive computation, video processing requires frequent memory access that consumes a large amount of power, limiting battery life and frustrating mobile users. On one hand, memory designers are focusing on hardware-level power-optimization techniques without considering how hardware performance influences viewers' actual experience. On the other hand, the human visual system is limited in its ability to detect subtle degradations in image quality; for example, under conditions of high ambient illumination, such as outdoors in direct sunlight, the veiling luminance (i.e., glare) on the screen of a mobile device can effectively mask imperfections in the image, so that under these circumstances a video can be rendered in lower than full quality without the viewer being able to detect any difference. This isolation between hardware design and viewer experience significantly increases hardware implementation overhead due to overly pessimistic design margins. This project integrates viewer-awareness and hardware adaptation to achieve power optimization without degrading video quality, as perceived by users. The results of this project will impact both basic research on hardware design and human vision, and provide critical viewer awareness data from human subjects, which can be used to engineer better video rendering for increased battery life on mobile devices. The project will directly involve undergraduate and graduate students, including females and Native Americans, in interdisciplinary research. Developing a viewer-aware mobile video-memory solution has proven to be a very challenging problem due to (i) complex existing viewer-experience models; (ii) memory modules without runtime adaptation; and (iii) the difficulty of viewer-experience analysis for hardware designers. This project addresses the problem by (i) focusing on the most influential viewing-context factor impacting viewer experience - ambient luminance; (ii) proposing novel methodologies for adaptive hardware design; and (iii) integrating a unique combination of expertise from the investigators, ranging from psychology to Integrated Circuit design and embedded systems. Specifically, this project will (i) experimentally and mathematically connect viewer experience, ambient illuminance, and memory performance; (ii) develop energy-quality adaptive hardware that can adjust memory usage based on ambient luminance so as to reduce power usage without impacting viewer experience; and (iii) design a mobile video system to fully evaluate the effectiveness of the developed methodologies.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

移动设备（例如智能手机）正在越来越多地用于观看视频，因为它们可以随时随地方便地用于此目的，例如在地铁或火车上通勤，坐在候诊室中或在家里闲逛。由于数据大小和密集的计算，视频处理需要频繁的内存访问，这会消耗大量功率，限制电池寿命和令人沮丧的移动用户。一方面，内存设计师专注于硬件级功率优化技术，而无需考虑硬件性能如何影响观众的实际体验。另一方面，人类视觉系统的检测能力限制了图像质量的微妙降解的能力。例如，在高环境照明的条件下，例如在阳光直射的户外，移动设备屏幕上的面纱亮度（即眩光）可以有效地掩盖图像中的缺陷，因此，在这种情况下，可以在视频中以低于全质量呈现视频，而观众无法检测任何差异。硬件设计与观看器体验之间的这种隔离大大增加了由于过度悲观的设计边缘而增加了硬件的开销。该项目集成了观看器的意识和硬件适应，以实现功率优化，而不会降低视频质量，这是用户所感知的。该项目的结果将影响有关硬件设计和人类视觉的基础研究，并提供来自人类受试者的关键观众意识数据，这些数据可用于设计更好的视频渲染，以增加移动设备上的电池寿命。该项目将直接参与包括女性和美洲原住民在内的本科生和研究生跨学科研究。由于（i）复杂的现有观众体验模型，开发一种观众感知的移动视频记忆解决方案已被证明是一个非常具有挑战性的问题； （ii）没有运行时适应的内存模块； （iii）观众体验分析对硬件设计师的困难。该项目通过（i）重点关注影响查看器体验的最有影响力的观看范围因素 - 环境亮度； （ii）提出自适应硬件设计的新方法； （iii）整合研究人员的专业知识的独特组合，从心理学到整合电路设计和嵌入式系统。具体而言，该项目将（i）实验和数学上连接查看器体验，环境照明和内存性能； （ii）开发能量质量的自适应硬件，该硬件可以根据环境亮度调整内存使用情况，以便在不影响观众体验的情况下降低功率使用； （iii）设计一种移动视频系统，以充分评估开发方法的有效性。该奖项反映了NSF的法定任务，并使用基金会的知识分子优点和更广泛的影响审查标准，被认为值得通过评估。