Advanced methods for depth-based human pose estimation and motion analysis: application to vital signs monitoring in the intensive care unit

基于深度的人体姿势估计和运动分析的先进方法：应用于重症监护病房的生命体征监测

• 批准号: RGPIN-2020-06695
• 负责人: Seoud, Lama
• 金额: $ 1.75万
• 依托单位: École Polytechnique de Montréal
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=754214
• 关键词: Advanced; methods; depth; based; human

Despite the mature and affordable solutions offered today in commoditized range sensors such as Microsoft's Kinect with its skeletal tracking algorithm, several challenges remain regarding depth-based human pose and motion analysis. Existing algorithms rely mostly on supervised learning with training data obtained in almost ideal conditions. Considerations with regards to range sensing technology, to subject's clothing and to visual occlusions are barely taken into account in the training. Another question remains regarding the performance of these methods when the subject is sitting or even lying down. Moreover, historically, the primary focus in human motion analysis has been on estimating and/or tracking human body joints over time. Despite its popularity, joint-based representation does not offer enough resolution to analyze fine and local motion such as human tremors, respiratory motion and nodding for example. There is a need in moving beyond the stick-figure view of the human body toward recovering richer descriptions of shape and motion. Building on my previous work, the long term objective of this research program is to propose novel dense representations of human pose and motion from depth images that allow a robust analysis of both fine-local and ample-global motion in real-time and in unconstrained environment. To achieve this, we will develop and validate novel computational tools and methods that address the challenges related to range sensor variability, to posture variability, to fine motion description and to severe occlusions. These tools will then be applied in the context of patients monitoring in the pediatric intensive care unit (PICU) for real-time detection of signs of vital distress. The program is composed of four specific objectives: (SO1) to enhance the robustness of depth-based human pose estimation to sensors specific artifacts and to bed-ridden postures, (SO2) to propose a new semantic dense motion descriptor from a sequence of depth images while taking into account the strategies developed in SO1, (SO3) to propose new strategies for human pose estimation and motion tracking in the presence of sever occlusions, (SO4) to detect abnormal head and limb movements in PICU patients by applying the tools developed in SO2 and SO3. These objectives will train 2 PhD, 3 master and 5 undergraduate students. By tackling important technical challenges, the proposed program will lead to major advances in human pose estimation and motion analysis. The computational tools will benefit a wide variety of applications such as robotics, surveillance, gaming and advanced manufacturing. In-bed motion analysis has received little interest so far although it brings innovative value into the market of commoditized child monitoring systems. Finally, this research program will have a direct impact on the quality of care in the PICU by preventing management delays, improving medical staff's efficiency and thus patients' outcome.

尽管今天在商品化范围传感器（例如Microsoft的Kinect及其骨骼跟踪算法）中提供了成熟且负担得起的解决方案，但在基于深度的人体姿势和运动分析方面仍然存在一些挑战。现有算法主要依赖于监督学习，并在几乎理想的条件下获得的培训数据。在培训中，几乎没有考虑到范围传感技术，受试者的服装和视觉遮挡的考虑。当受试者坐着甚至躺下时，这些方法的性能仍然存在另一个问题。此外，从历史上看，人类运动分析的主要重点一直是随着时间的推移估计和/或跟踪人体关节。尽管它很受欢迎，但基于联合的表示并未提供足够的分辨率来分析诸如人类震颤，呼吸运动和点头等局部运动。有必要超越人体的棍子数字，以恢复形状和运动的更丰富的描述。 在我以前的工作的基础上，该研究计划的长期目标是提出从深度图像中提出的新型人类姿势和运动的致密表示，这些图像可以实时和无限制地对精美的局部和全球运动进行良好的分析环境。为了实现这一目标，我们将开发并验证新型的计算工具和方法，以应对与范围传感器可变性，姿势变异性，精细运动描述和严重闭塞相关的挑战。然后，这些工具将在儿科重症监护病房（PICU）监测的患者监测的情况下应用，以实时检测至关重要的迹象。 该程序由四个特定目标组成：（SO1）增强基于深度的人类姿势估计对传感器特定伪像和卧床式姿势的鲁棒性，（SO2），从一系列深度提出新的语义密集运动描述符图像在考虑到SO1中开发的策略时，（SO3）在存在性闭塞的存在下提出新的姿势估算和运动跟踪策略在SO2和SO3中。这些目标将培训2位博士学位，3名硕士和5名本科生。 通过应对重要的技术挑战，该计划将导致人类姿势估计和运动分析的重大进展。计算工具将受益于多种应用，例如机器人技术，监视，游戏和高级制造。迄今为止，内部运动分析几乎没有兴趣，尽管它将创新价值带入了商品化儿童监测系统的市场。最后，该研究计划将通过防止管理延误，提高医务人员的效率，从而直接影响PICU的护理质量。