AR-integrated intelligent visual inspection system for health monitoring of constructed facilities

AR集成智能视觉检测系统，用于建筑设施的健康监测

• 批准号: RGPIN-2022-05151
• 负责人: Chen, PoHan
• 金额: $ 2.26万
• 依托单位: Concordia University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=746377
• 关键词: AR; integrated; intelligent; visual; inspection

The aging of constructed facilities, such as buildings, bridges, pipelines, etc., is a critical issue worldwide, as it affects the safety of the facilities. In Canada, many constructed facilities, especially infrastructure facilities, are in sparsely populated or indigenous areas, and the development of an automated intelligent visual inspection system that can greatly reduce manpower requirement and enhance inspection efficiency and effectiveness is necessary. In this regard, the goal of this research program is to develop near-real-time (NRT) fast defect recognition algorithms and utilize augmented reality (AR) to develop an AR-integrated intelligent visual inspection system for near-real-time health monitoring of constructed facilities. This intelligent system will comprise two modes: (1) automated drone inspection mode (M1); and (2) in-person inspection mode (M2). For M1, a GPS-guided drone will fly along a pre-planned route to automatically capture facility surface inspection data (e.g., videos or photos), and display them on a tablet or smartphone in real time, followed by immediate superimposing of recognized defects on the displayed view (i.e., AR effects). Whenever a defect is identified, the corresponding GPS position will be recorded. After the drone finishes the entire planned route, it will automatically fly back to each of the recorded positions to capture more detailed data around the spotted defect. For M2, an AR-HMD (head-mounted display) will capture inspection data and transmit them to a laptop for fast defect recognition, followed by superimposing recognized defects on AR-HMD. Interactive gesture control of AR-HMD will be incorporated in the development. To achieve the goal of this program, four research objectives are set: (1) to develop near-real-time fast defect recognition algorithms for steel and reinforced concrete (RC) constructed facilities; (2) to integrate a drone with tablets/smartphones of both iOS/iPadOS and Android systems for the M1 mode; (3) to integrate an AR-HMD and its gesture control with a laptop for the M2 mode; and (4) to design and develop user-friendly AR user interfaces for both modes. Nine Highly Qualified Personnel (HQP) will be trained in this program, including 3 PhD students, 4 MASc students, and 2 undergraduate research assistants (URAs). Nowadays, the AR applications in the Architecture, Engineering and Construction (AEC) industry mostly rely on pre-loaded data on the AR equipment for object or pattern recognition. This program will bring AR applications to the next level by processing newly captured data in a near-real-time manner, which leads to the importance of the fast defect recognition algorithms to be developed. At the same time, the proposed automated drone inspection mode and the in-person inspection mode with interactive AR-HMD may both revolutionize the current inspection practices and significantly benefit the public and private engineering sectors in Canada and beyond.

建筑物、桥梁、管道等建筑设施的老化是世界范围内的一个关键问题，因为它影响设施的安全。在加拿大，许多建筑设施，尤其是基础设施，都位于人烟稀少或原住民地区，因此有必要开发一种自动化智能视觉检测系统，可以大大减少人力需求，提高检测效率和效果。在这方面，本研究计划的目标是开发近实时（NRT）快速缺陷识别算法，并利用增强现实（AR）开发集成AR的智能视觉检查系统，用于近实时健康监测的已建设施。该智能系统将包含两种模式：（1）自动化无人机巡检模式（M1）； （2）现场检查方式（M2）。 对于M1，GPS引导的无人机将沿着预先计划的路线飞行，自动捕获设施表面检查数据（例如视频或照片），并将其实时显示在平板电脑或智能手机上，然后立即叠加已识别的缺陷在显示的视图上（即 AR 效果）。每当发现缺陷时，就会记录相应的 GPS 位置。无人机完成整个计划路线后，会自动飞回每个记录的位置，以捕获发现的缺陷周围的更详细的数据。对于M2，AR-HMD（头戴式显示器）将捕获检查数据并将其传输到笔记本电脑以进行快速缺陷识别，然后将识别到的缺陷叠加在AR-HMD上。 AR-HMD的交互式手势控制将纳入开发中。为了实现该计划的目标，设定了四个研究目标：（1）开发钢和钢筋混凝土（RC）建筑设施的近实时快速缺陷识别算法； （2）将无人机与iOS/iPadOS和Android系统的平板电脑/智能手机集成，实现M1模式； (3) 将AR-HMD及其手势控制与笔记本电脑集成以用于M2模式； (4) 为两种模式设计和开发用户友好的 AR 用户界面。该项目将培养9名高素质人才（HQP），其中博士生3名，硕士生4名，本科生研究助理（URA）2名。如今，建筑、工程和施工 (AEC) 行业的 AR 应用主要依赖于 AR 设备上预加载的数据来进行对象或模式识别。该计划将通过以近实时的方式处理新捕获的数据，将 AR 应用提升到一个新的水平，这导致了要开发的快速缺陷识别算法的重要性。与此同时，拟议的自动化无人机检查模式和交互式 AR-HMD 的现场检查模式都可能彻底改变当前的检查实践，并显着使加拿大及其他地区的公共和私营工程部门受益。