FW-HTF-R/Collaborative Research: Human-Robot Sensory Transfer for Worker Productivity, Training, and Quality of Life in Remote Undersea Inspection and Construction Tasks

FW-HTF-R/合作研究：人机感官传递可提高远程海底检查和施工任务中工人的生产力、培训和生活质量

• 批准号: 2128924
• 负责人: Michael Krieg
• 金额: $ 40.5万
• 依托单位: University of Hawaii
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-12-01 至 2025-11-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2128924&HistoricalAwards=false
• 关键词: FW; HTF; Collaborative; Research; Human

This Future of Work at the Human-Technology Frontier: Core Research project will create a novel interface for remote operation of undersea robots and customize it to the needs of offshore industries and workers. The novel interface integrates robot sensor readings and high-speed predictive simulations of hydrodynamic forces to create an immersive mixed reality (MR) display. In addition to augmented video images of the robot's surroundings, the interface converts measurements of water flow rates, hydrostatic pressure, ambient temperature, and other variables into tactile sensations for the operator. Likewise, the interface will render natural movements of the operator's body into control commands to the robot. The goal is human-robot "sensory transfer," that is, seamless translation of perceptions and actions between the operator and the robot. The goal of this project is to develop and match the capabilities of the interface to industry and worker needs. One anticipated benefit is to reduce the extensive training currently required for operators, thereby increasing access to these jobs while reducing industry training expenses and downtime due to personnel shortages. The project will study the most effective way to improve worker performance, safety, and quality of life, and by requiring a diverse set of subjects, will show how such human-robot interfaces can expand economic opportunity to broad sections of society. The interface can also be used in a purely virtual mode as a training tool. The project will examine the use of this capability to recruit workers from adjacent fields, such as construction. Offshore applications that would directly benefit from this project include subsea infrastructure inspection, geological surveys, marine habitat monitoring, pollution assessments, ship-hull inspections, unexploded ordnance surveys, contraband detection, aquaculture monitoring, search and rescue, and archaeological exploration and surveys. An increase in extreme weather and rising sea levels will place increasing demands on offshore operations to protect and repair coastal damage. Similarly offshore sustainable energy infrastructure such as wind, wave, or tidal generators will increase the demand for undersea inspection, construction, and maintenance.This project will reconceptualize future subsea industry by advancing knowledge of underwater Human-Robot Interaction (HRI) in under-explored subsea workplaces, illuminating socioeconomic features and adult-learning needs of workforce transformation to subsea industry, and establishing academia-industry-government partnerships for improving performance, safety, and societal outcomes of subsea works. Novel human-robot sensory transfer methods are suggested for reliability against conditions unique to subsea. These methods will support fast and accurate reconstruction of subsea workplaces. MR will be used to generate human-perceivable simulation of remote subsea workplaces in real time based on feedback from a novel robotic sensing and data transmission system. Motion capture will be created for easier navigation of remotely operated vehicles (ROVs). This research will establish new knowledge on motivational and educational determinants of introducing easy-to-use collaborative ROVs as part of a transformative workforce for future subsea robot operations, through extensive participation from industrial partners. The assessment will integrate techniques from psychometric and behavioral sciences as well as engineering and human factors. The work will also pioneer the development of a future subsea job framework for integration of ROVs into a participatory delivery of core subsea services. The economic benefits of robotic adoption will be estimated based on demand projection and elasticity estimation. This research will transform the frontiers of human-technology partnership in the context of the future subsea industry, reposition workforce threatened by automation in other domains, enhance future workers’ safety and well-being, and improve subsea operation performance, thus enhancing the long-term sustainable ocean exploration.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

人类技术领域的工作未来：核心研究项目将创建一个新颖的界面，用于远程操作机器人的远程操作，并根据离岸行业和工人的需求自定义它。新颖的界面集成了机器人传感器读数和流体动力的高速预测模拟，以创建沉浸式混合现实（MR）显示。除了增强机器人周围环境的视频图像外，该界面还将水流量，静水压力，环境温度和其他变量的测量转换为操作员的触觉感觉。同样，该界面将使操作员的身体自然移动到机器人的控制命令中。目标是人类机器人“感觉转移”，即对操作员和机器人之间的看法和动作的无缝翻译。该项目的目的是开发和匹配界面的功能与行业和工人需求。一个预期的好处是减少运营商当前所需的广泛培训，从而增加对这些工作的机会，同时减少由于人员短缺而减少行业培训费用和停机时间。该项目将研究提高工人绩效，安全性和生活质量的最有效方法，并通过需要一组潜水员来展示这种人类机器人界面如何将经济机会扩大到社会的广泛阶层。该界面也可以在纯虚拟模式下用作训练工具。该项目将研究这种能力从相邻领域（例如建筑）招募工人的使用。将直接从该项目中受益的离岸应用包括海底基础设施检查，地质调查，海洋栖息地监测，污染评估，船舶救护者检查，未爆炸的军械调查，违禁品检测，水产养殖监测，搜索和救援，以及考古学探索以及考古学勘探和潮流。极端天气和海平面上升的增加将使对保护和修复沿海破坏的海上行动的需求不断增加。 Similarly offshore sustainable energy infrastructure such as wind, wave, or tidal generators will increase the demand for undersea inspection, construction, and maintenance.This project will reconceptualize future subsea industry by advancing knowledge of underwater Human-Robot Interaction (HRI) in under-explored subsea Workplaces, illuminating socioeconomic features and adult-learning needs of workforce transformation to subsea industry, and establishing学术界与政府的合作伙伴关系，以改善海底工作的绩效，安全和社会成果。建议使用新型的人类机器人感觉传递方法针对海底独有的条件进行可靠性。这些方法将支持海底工作场所的快速准确重建。 MR将根据新型的机器人灵敏度和数据传输系统的反馈来实时对远程海底工作场所的人类可可仿真。将创建运动捕获，以便更轻松地导航远程操作的车辆（ROV）。这项研究将通过引入易于使用的协作ROV的动机和教育决定者建立新的知识，这是通过工业合作伙伴的广泛参与，作为未来海底机器人运营的变革性劳动力的一部分。评估将整合心理学和行为科学以及工程和人为因素的技术。这项工作还将开拓未来的海底工作框架，以将ROV集成到核心海底服务的参与交付中。机器人采用的经济利益将根据需求预测和弹性估计来估算。这项研究将在未来海底行业的背景下改变人类技术伙伴关系的前沿，由其他领域的自动化受到威胁，增强未来工人的安全和福祉，并提高海底运营绩效，从而提高长期可持续可持续的海洋探索。从而通过评估了NSF的法规和范围，这表明了NSF的审查范围，这是通过评估的范围来宣传的。

项目成果

VR-Based Haptic Simulator for Subsea Robot Teleoperations

• DOI: 10.1061/9780784483893.126
• 发表时间: 2022-05
• 期刊: Computing in Civil Engineering 2021
• 作者: Fang Xu;Qi Zhu;Shuai Li;Zhuoyuan Song;Jing Du

Virtual Telepresence for the Future of ROV Teleoperations: Opportunities and Challenges

ROV 远程操作未来的虚拟远程呈现：机遇与挑战

• DOI: 10.5957/tos-2022-015
• 发表时间: 2022
• 期刊: The SNAME 27th Offshore Symposium
• 作者: Xia, Pengxiang;McSweeney, Kevin;Wen, Feng;Song, Zhuoyuan;Krieg, Michael;Li, Shuai;Yu, Xiao;Crippen, Kent;Adams, Jonathan;Du, Eric Jing
• 通讯作者: Du, Eric Jing