Actor-Critic-Like Stochastic Adaptive Search Algorithms for Simulation Optimization

用于仿真优化的类似 Actor-Critic 的随机自适应搜索算法

• 批准号: 1634627
• 负责人: Jiaqiao Hu
• 金额: $ 19.99万
• 依托单位: SUNY at Stony Brook
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-09-01 至 2019-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1634627&HistoricalAwards=false
• 关键词: Actor; Critic; Like; Stochastic; Adaptive

Many systems arising in applications from engineering design, manufacturing, and health care require the use of simulation optimization techniques to improve their performance. However, despite significant progress in recent years, simulation optimization remains an area with many theoretical and practical challenges. This research project aims to expand the current knowledge in this field by investigating a novel approach that integrates theories and tools from reinforcement learning (a subarea of artificial intelligence) within a class of adaptive search algorithms called the model-based methods to solve simulation optimization problems. Because of the generality of these methodologies, the resulting techniques will have broad applicability in a wide array of industry and science sectors. In particular, through collaboration with power engineers, the developed algorithms will be tested and applied to voltage control problems in electric power systems, potentially benefiting both utility companies and energy consumers. The research plan will be closely integrated with the education and training of students in engineering by incorporating new developments into the graduate courses the investigator teaches and recruiting female and underrepresented minority students to the project.The goal of this research is to advance theoretical underpinnings of new model-based algorithms that can be orders of magnitude more efficient than the state-of-the-art. This will be accomplished by exploring the connections between model-based methods and policy gradient-based reinforcement-learning algorithms. Specifically, the investigator will examine how to use the insights from actor-critic algorithms in the reinforcement learning framework to effectively reduce the sampling variance of model-based methods. If successful, the approach will integrate function approximation techniques within a model-based optimization setting to provide algorithms with low-variance performance estimates in searching for improved solutions. This research may change the manner in which these algorithms are implemented and applied, leading to faster and more efficient algorithms for solving a broad class of optimization problems, especially in settings that require expensive function evaluations or simulations for performance estimation.

工程设计、制造和医疗保健等应用中出现的许多系统都需要使用仿真优化技术来提高其性能。然而，尽管近年来取得了重大进展，仿真优化仍然是一个面临许多理论和实践挑战的领域。该研究项目旨在通过研究一种新颖的方法来扩展该领域的当前知识，该方法将强化学习（人工智能的一个子领域）的理论和工具集成到一类称为基于模型的方法的自适应搜索算法中，以解决模拟优化问题。由于这些方法的通用性，由此产生的技术将在广泛的工业和科学领域具有广泛的适用性。特别是，通过与电力工程师的合作，开发的算法将被测试并应用于电力系统中的电压控制问题，这可能使公用事业公司和能源消费者受益。该研究计划将通过将新的发展纳入研究者教授的研究生课程中，并招募女性和代表性不足的少数族裔学生参与该项目，与工程专业学生的教育和培训紧密结合。这项研究的目标是推进新的理论基础基于模型的算法，其效率比最先进的算法高几个数量级。这将通过探索基于模型的方法和基于策略梯度的强化学习算法之间的联系来实现。具体来说，研究人员将研究如何在强化学习框架中使用演员批评算法的见解来有效减少基于模型的方法的采样方差。如果成功，该方法将在基于模型的优化设置中集成函数逼近技术，为算法提供低方差性能估计，以搜索改进的解决方案。这项研究可能会改变这些算法的实现和应用方式，从而产生更快、更有效的算法来解决广泛的优化问题，特别是在需要昂贵的函数评估或模拟来进行性能估计的情况下。

项目成果

A minimum deviation approach for improving the consistency of uncertain 2-tuple linguistic preference relations

• DOI: 10.1016/j.cie.2018.01.024
• 发表时间: 2018-03
• 期刊: Comput. Ind. Eng.
• 作者: Shengbao Yao;Jiaqiao Hu

A two-time-scale adaptive search algorithm for global optimization

• DOI: 10.1109/wsc.2017.8247940
• 发表时间: 2017-12
• 期刊: 2017 Winter Simulation Conference (WSC)
• 作者: Qi Zhang-;Jiaqiao Hu

Some Monotonicity Results for Stochastic Kriging Metamodels in Sequential Settings

序列设置中随机克里金元模型的一些单调性结果

• DOI: 10.1287/ijoc.2017.0779
• 发表时间: 2018
• 期刊: INFORMS Journal on Computing
• 影响因子: 2.1
• 作者: Wang, Bing;Hu, Jiaqiao
• 通讯作者: Hu, Jiaqiao

Surrogate-Based Promising Area Search for Lipschitz Continuous Simulation Optimization

基于替代的有前景区域搜索 Lipschitz 连续仿真优化

• DOI: 10.1287/ijoc.2017.0801
• 发表时间: 2018
• 期刊: INFORMS Journal on Computing
• 影响因子: 2.1
• 作者: Fan, Qi;Hu, Jiaqiao
• 通讯作者: Hu, Jiaqiao

Enhancing Random Search with Surrogate Models for Lipschitz Continuous Optimization

使用 Lipschitz 连续优化的替代模型增强随机搜索

• DOI: 10.1109/coase.2019.8843031
• 发表时间: 2019
• 期刊: 2019 IEEE 15th International Conference on Automation Science and Engineering (CASE
• 作者: Zhang, Qi;Hu, Jiaqiao
• 通讯作者: Hu, Jiaqiao