Derivative-Free Optimization: Algorithmic Developments, Software Design, Applications, and Machine Learning

无导数优化：算法开发、软件设计、应用程序和机器学习

• 批准号: RGPIN-2018-05286
• 负责人: LeDigabel, Sébastien
• 金额: $ 3.13万
• 依托单位: École Polytechnique de Montréal
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2019
• 资助国家: 加拿大
• 起止时间: 2019-01-01 至 2020-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=688610
• 关键词: Derivative; Free; Optimization; Algorithmic; Developments

My research focuses on derivative-free optimization (DFO), which is essential in many engineering applications. More precisely, I am concerned with blackbox optimization, which occurs when the objective(s) and constraints of an engineering optimization problem are obtained by a computer code seen as a blackbox. Such blackboxes may be expensive to evaluate, may be contaminated with noise, and sometimes fail to return a value. In this context, I am considering the mesh adaptive direct search algorithm (MADS). This proposal is the continuation of my previous NSERC Discovery Grant and describes a five-year research program based on extensions of MADS and the links between DFO and machine learning (ML). It is divided into 12 projects that are well-suited for students, who will develop advanced skills in both optimization and ML.******The program will address algorithm design and analysis, as well as software development and applications to various fields.******The algorithmic projects will develop new tools that will enhance the solution of blackbox optimization problems. They include the introduction of new surrogate techniques based on ML; sensitivity analyses to identify and scale the most important variables; robust optimization methods for constrained noisy problems; and a new multiobjective algorithm based on the use of different measures of the quality of an approximate Pareto front. Another project involves ML to tune the MADS parameters. The new algorithmic features will be mathematically analyzed to prove convergence.******A major part of my research is the design and dissemination of free software. In particular, the state-of-the-art NOMAD package that I have developed since 2008 is freely available under the LGPL licence at www.gerad.ca/nomad. NOMAD is in constant evolution, and the algorithmic developments will be integrated into the package by the students in charge of the projects with the help of two research associates funded from other sources. The proposed research will also provide new versions of the sgtelib library of surrogates, and two application codes for benchmarking purposes in the DFO community.******The last component of my research concerns real engineering applications. For example, in the past, I worked on the optimization of alloy design, aircraft design, and energy. In this proposal, we will use applications in material science to test new surrogates; develop robust solutions for noisy mechanical engineering problems; consider electrical engineering problems with two or three objectives; develop a solar farm simulator that will include most of the typical blackbox-problem characteristics. Finally, we will apply DFO techniques to the optimization of the hyperparameters of deep neural networks.******This proposal contributes to the training of 9 graduate and undergraduate students who will receive a multidisciplinary formation with fundamental and applied aspects.

我的研究重点是无衍生优化（DFO），这在许多工程应用中至关重要。更确切地说，我关注的是黑框优化，这是在目标（S）和工程优化问题的约束时发生的。这样的黑盒可能很昂贵，可能会被噪音污染，有时无法返回值。在这种情况下，我正在考虑网格自适应直接搜索算法（MADS）。该提案是我以前的NSERC Discovery Grant的延续，并根据MAD的扩展以及DFO与机器学习（ML）之间的联系描述了一项为期五年的研究计划。它分为12个项目，非常适合学生，他们将开发优化和ML的先进技能。******该计划将解决算法设计和分析，以及软件开发和针对各个领域的应用程序。********算法项目将开发新的工具，以增强黑盒优化问题的解决方案。其中包括基于ML的新代孕技术的引入；灵敏度分析以识别和扩展最重要的变量；有限的嘈杂问题的强大优化方法；以及一种基于近似帕累托前部质量的不同度量的新型多目标算法。另一个项目涉及ML调整MADS参数。新的算法功能将进行数学分析以证明融合。******我的研究的主要部分是自由软件的设计和传播。特别是，自2008年以来，我开发的最先进的游牧套件可在www.gerad.ca/nomad的LGPL许可下免费获得。 Nomad持续发展，算法开发将由负责项目的学生纳入包装，并在两位由其他来源资助的研究协会的帮助下。拟议的研究还将提供SGTELIB库库的新版本，以及在DFO社区中用于基准目的的两个应用程序代码。******我的研究的最后一个组成部分涉及真正的工程应用程序。例如，过去，我从事合金设计，飞机设计和能源的优化。在此提案中，我们将使用材料科学中的应用来测试新的代孕。为嘈杂的机械工程问题开发强大的解决方案；考虑两个或三个目标的电气工程问题；开发一个太阳能农场模拟器，其中包括大多数典型的黑盒问题特征。最后，我们将将DFO技术应用于深度神经网络的超参数的优化。