CRII: CSR: Pervasive Gesture Recognition Using Ambient Light

CRII：CSR：使用环境光进行普遍手势识别

• 批准号: 1565609
• 负责人: Muhammad Shahzad
• 金额: $ 17.49万
• 依托单位: North Carolina State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-05-01 至 2020-04-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1565609&HistoricalAwards=false
• 关键词: CRII; CSR; Pervasive; Gesture; Recognition

As computing devices are becoming smaller, smarter, and more ubiquitous, computing has started to embed into our environment in various forms such as intelligent thermostats, smart appliances, remotely controllable household equipment, and weather based automated lawn irrigation systems. Consequently, we need new ways to seamlessly and effectively communicate and interact with such ubiquitous and always-available computing devices. A natural choice for such communication and interaction is human gestures because gestures are an integral part of the way humans communicate and interact with each other in their daily lives. This project aims at using ambient light and cheap commercial off-the-shelf light sensors to develop a gesture recognition system. The intuition behind this approach is that as a user performs a gesture in a room that is lit with light, the amount of light that he/she reflects and blocks changes, resulting in changes in the intensity of light in all parts of the room. The patterns of change in the intensity of light are different for different gestures, which can be learnt and used to recognize the gestures.In developing the ambient light based gesture recognition system, this project has two primary objectives: (1) environment independence, i.e., making the system agnostic to the characteristics of the environment, such as different lighting conditions, and (2) user independence, i.e., making the system agnostic to the number of users in a room and their routine activities. Several challenges arise in developing the ambient light based gesture recognition system, such as automatically detecting the start and end of a gesture, removing noise from the time-series of sensor values, handling the varying time durations of the different occurrences of the same gesture, simultaneously recognizing the gestures of multiple people, and recognizing the gestures of non-stationary users. This project will not only address these and other similar challenges, but will also advance the knowledge and understanding of the use of ambient light for novel systems by yielding a theoretical foundation for modeling human gestures and routine activities using changes in the intensity of light.The successful completion of this project will greatly benefit our society. First, this project will make the data set collected during this project publicly available for research. Second, the proposed ambient light based gesture recognition system will introduce a new and convenient way for users to interact with the computing embedded in their environments. Third, the proposed project will bridge several different communities such as systems, signal processing, machine learning, mobile computing, and human computer interaction; and foster interaction and communication among them. Fourth, the educational side of the project will integrate the research findings into the undergraduate and graduate curricula at North Carolina State University.

随着计算设备变得越来越小、越来越智能、越来越普遍，计算已经开始以各种形式嵌入到我们的环境中，例如智能恒温器、智能电器、远程控制家用设备和基于天气的自动草坪灌溉系统。因此，我们需要新的方法来与这种无处不在且始终可用的计算设备进行无缝、有效的通信和交互。这种交流和互动的自然选择是人类手势，因为手势是人类在日常生活中相互交流和互动方式不可或缺的一部分。该项目旨在使用环境光和廉价的商用现成光传感器来开发手势识别系统。这种方法背后的直觉是，当用户在有光的房间中执行手势时，他/她反射和阻挡的光量会发生变化，从而导致房间所有部分的光强度发生变化。不同手势的光强度变化模式不同，可以学习并用于识别手势。 在开发基于环境光的手势识别系统时，该项目有两个主要目标：（1）环境独立性，即。 ，使系统与环境特征无关，例如不同的照明条件，以及（2）用户独立性，即使系统与房间中的用户数量及其日常活动无关。在开发基于环境光的手势识别系统时出现了一些挑战，例如自动检测手势的开始和结束、从传感器值的时间序列中消除噪声、处理同一手势不同发生的不同持续时间、同时识别多人的手势，并识别非静止用户的手势。该项目不仅将解决这些和其他类似的挑战，还将通过利用光强度的变化对人类手势和日常活动进行建模提供理论基础，从而增进对新颖系统使用环境光的认识和理解。该项目的成功完成将极大造福我们的社会。首先，该项目将公开该项目期间收集的数据集以供研究。其次，所提出的基于环境光的手势识别系统将为用户引入一种新的、便捷的方式与嵌入在其环境中的计算进行交互。第三，拟议项目将连接系统、信号处理、机器学习、移动计算和人机交互等多个不同社区；并促进他们之间的互动和沟通。第四，该项目的教育方面将把研究成果整合到北卡罗来纳州立大学的本科生和研究生课程中。