NRI: FND: Intelligent Co-robots for Complex Welding Manufacturing through Learning and Generalization of Welders Capabilities

NRI：FND：通过学习和推广焊工能力实现复杂焊接制造的智能协作机器人

• 批准号: 2024614
• 负责人: YuMing Zhang
• 金额: $ 66.55万
• 依托单位: University of Kentucky Research Foundation
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-08-01 至 2024-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2024614&HistoricalAwards=false
• 关键词: NRI; FND; Intelligent; Co; robots

There is a dramatic and growing shortage of highly skilled welders, accentuated by the fact that manufacturing complexity and production volumes are rising. As a result, global use of robotic welding is expanding rapidly. However, current welding robots are not as adaptive and creative as human welders in performing complex welding tasks that require sophisticated skills. This award supports fundamental research on advancing the robotic capabilities needed to realize fully robotic automation of complex welding tasks. The research will endow collaborative welding robots with sophisticated welding knowledge, expert intelligence, and an interactive learning capability to enable them to address dynamic welding scenarios. The research results will both enhance the scientific base for robotic control and facilitate the realization of fully automatic, robotic, and intelligent manufacturing. The research involves several disciplines, including welding, process monitoring, data visualization, machine learning, optimization, and robotic control. That multi-disciplinary approach will broaden the participation of students from diverse backgrounds in research, and the knowledge gained will be incorporated in curricula in robotic and intelligent manufacturing. The double-electrode, gas metal arc welding process is complex, requiring intense collaboration between expert welders. As a result, the robotic automation of such a complex welding process requires advances in the scientific base of robotic perception, learning, and control. The project will research advanced methods for the extraction of expert welding-domain knowledge and the quantification and interpretation of that knowledge for use by collaborative robots, thereby equipping collaborative welding robots to perform complex welding tasks. To realize that goal, the research team will: 1) build an immersive virtual reality system with a three-dimensional rendering of the weld pool and arc that can characterize the weld scene and record human operations, 2) use an explainable recurrent convolutional neural network to perform causal analysis of the torch manipulation of human welders to obtain its relationship to dynamic weld pool/arc evolution, 3) generalize the results in terms of human heterogeneity by using transfer learning to extract common latent knowledge from different human welders, and 4) develop an interactive learning module that allows collaborative robots to be supervised by on-site human welders through the reinforcement learning-based perception of language instructions and human gestures.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

高技能焊机的短缺且越来越大，这一事实强调了制造的复杂性和生产量正在上升。结果，机器人焊接的全球使用正在迅速扩展。 但是，目前的焊接机器人在执行需要复杂技能的复杂焊接任务时不如人类焊工那么自适应和创造力。该奖项支持推进实现复杂焊接任务完全机器人自动化所需的机器人功能的基础研究。这项研究将赋予具有复杂的焊接知识，专家智能和互动学习能力的协作焊接机器人，以使其能够解决动态焊接方案。研究结果既可以增强机器人控制的科学基础，又可以促进实现全自动，机器人和智能制造的实现。该研究涉及多个学科，包括焊接，过程监控，数据可视化，机器学习，优化和机器人控制。这种多学科的方法将扩大来自不同背景研究的学生的参与，并且获得的知识将在机器人和智能制造业中纳入课程中。 双电极，气体金属电弧焊接过程很复杂，需要专家焊工之间的强烈协作。 结果，这种复杂的焊接过程的机器人自动化需要在机器人感知，学习和控制的科学基础上取得进步。该项目将研究用于提取专家焊接域知识的高级方法，以及对协作机器人使用该知识的量化和解释，从而为协作式焊接机器人配置以执行复杂的焊接任务。为了意识到这一目标，研究团队将：1）建立一个具有三维焊池和电弧的三维渲染，可以表征焊接场景并记录人类操作，2）使用可解释的卷积神经网络来执行对人类焊接的触发性，以使其对人类的关系进行构成型号的关系，以使其对人的关系进行分析，以使其与人的关系逐步化，以使其逐渐逐步化量子，以使台化的量化量/台化的效果，3） using transfer learning to extract common latent knowledge from different human welders, and 4) develop an interactive learning module that allows collaborative robots to be supervised by on-site human welders through the reinforcement learning-based perception of language instructions and human gestures.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

项目成果

Data-driven process characterization and adaptive control in robotic arc welding

• DOI: 10.1016/j.cirp.2022.04.046
• 发表时间: 2022-05
• 期刊: CIRP Annals
• 作者: Peng Wang;J. Kershaw;Matthew Russell;Jianjing Zhang;Yuming Zhang;R. X. Gao

Do We Need a New Foundation to Use Deep Learning to Monitor Weld Penetration?

• DOI: 10.1109/lra.2023.3270038
• 发表时间: 2023-06
• 期刊: IEEE Robotics and Automation Letters
• 影响因子: 5.2
• 作者: Edison Mucllari;Rui Yu;Yue Cao;Qiang Ye;Yuming Zhang

How to Accurately Monitor the Weld Penetration From Dynamic Weld Pool Serial Images Using CNN-LSTM Deep Learning Model?

• DOI: 10.1109/lra.2022.3173659
• 发表时间: 2022-07
• 期刊: IEEE Robotics and Automation Letters
• 影响因子: 5.2
• 作者: Rui Yu;J. Kershaw;Peng Wang;Yuming Zhang

Hybrid machine learning-enabled adaptive welding speed control

• DOI: 10.1016/j.jmapro.2021.09.023
• 发表时间: 2021-11
• 期刊: Journal of Manufacturing Processes
• 影响因子: 6.2
• 作者: J. Kershaw;Rui Yu;Yuming Zhang;Peng Wang

Monitoring of Backside Weld Bead Width from High Dynamic Range Images Using CNN Network

• DOI: 10.1109/codit55151.2022.9804041
• 发表时间: 2022-05
• 期刊: 2022 8th International Conference on Control, Decision and Information Technologies (CoDIT)
• 作者: Rui Yu;J. Kershaw;Peng Wang;Yuming Zhang