Collaborative Research: Continuous-State Reinforcement Learning for Remanufacturing

协作研究：再制造的连续状态强化学习

• 批准号: 2027527
• 负责人: Jiaqiao Hu
• 金额: $ 24.37万
• 依托单位: SUNY at Stony Brook
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-07-01 至 2025-06-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2027527&HistoricalAwards=false
• 关键词: Collaborative; Research; Continuous; State; Reinforcement

This award will contribute to the national prosperity and U.S. manufacturing competitiveness by developing new reinforcement learning (a subfield of artificial intelligence) methods to address inventory-control problems arising in remanufacturing industry. Remanufacturing is a product-management manufacturing process that aims to reduce the energy consumption and carbon footprint of traditional manufacturing. Effective production/inventory management to match the supply with the demand is a key element to the success of remanufacturing industry. However, the complexity of such problems and the uncertainties involved in the remanufacturing process make the conventional production planning methods difficult to apply. The resulting algorithms and tools will be fully tested using real-world data collected from the industry and are expected to achieve significant savings in raw materials and energy resources, leading to practical management policies of industrial interest. The PIs will involve both graduate and undergraduate students in this research and incorporate case studies into the advanced courses taught at different institutions.This research will be based on a fusion of techniques from reinforcement learning and the field of simulation optimization. Through novel adaptations of the-state-of-the-art variance reduction and function approximation techniques from simulation optimization, the PIs will investigate a new class of learning techniques especially tailored to remanufacturing decision-making problems. These include an extension of classical Q-learning for solving continuous-state semi-Markov decision processes and more general gradient-free actor-critic-like algorithms that overcome the local convergence of existing approaches. The algorithms developed will be studied for their theoretical properties such as convergence and performance consistency, and then assessed and validated on remanufacturing simulation models built on real-world data to investigate their practical impact.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该奖项将通过开发新的强化学习（人工智能的一个子领域）方法来解决再制造行业中出现的库存控制问题，从而为国家繁荣和美国制造业竞争力做出贡献。再制造是一种产品管理制造过程，旨在减少传统制造的能源消耗和碳足迹。有效的生产/库存管理以匹配供应与需求是再制造行业成功的关键要素。然而，此类问题的复杂性以及再制造过程中涉及的不确定性使得传统的生产计划方法难以应用。 由此产生的算法和工具将使用从行业收集的真实数据进行全面测试，预计将显着节省原材料和能源，从而制定符合行业利益的实用管理政策。 PI 将让研究生和本科生参与这项研究，并将案例研究纳入不同机构教授的高级课程中。这项研究将基于强化学习和模拟优化领域技术的融合。通过对最先进的方差减少和模拟优化的函数逼近技术进行新颖的改造，PI 将研究一类专门针对再制造决策问题而定制的新型学习技术。其中包括用于解决连续状态半马尔可夫决策过程的经典 Q 学习的扩展，以及克服现有方法的局部收敛性的更通用的无梯度行动评论家算法。所开发的算法将研究其收敛性和性能一致性等理论特性，然后在基于真实世界数据的再制造仿真模型上进行评估和验证，以研究其实际影响。该奖项反映了 NSF 的法定使命，并被认为是值得的通过使用基金会的智力优势和更广泛的影响审查标准进行评估来提供支持。