RI:Small:Robust Performance Models

RI：小型：稳健的性能模型

• 批准号: 1813537
• 负责人: Lars Kotthoff
• 金额: $ 41.2万
• 依托单位: University of Wyoming
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-09-01 至 2023-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1813537&HistoricalAwards=false
• 关键词: RI; Small; Robust; Performance; Models

Algorithms are ubiquitous in modern society and integral to the economy. Whether finding optimal assignments of packages and operators to planes, trucks, and ships, or translating between different languages, the problems solved become larger and more challenging every day. Crucially enabling such developments are advances in artificial intelligence. There are often different approaches for solving the same type of problem, and they are often synergistic -- where one fails, another performs well. AI techniques in this project allow the best approach for a given problem to be chosen automatically. This research will allow for such choices to be made more robustly even in difficult circumstances, resulting in improved performance and reduced effort to deploy AI in practical systems. Ultimately, the project will make it easier for humans to develop high-performance AI systems.Algorithm selection is the process of automatically matching synergistic algorithmic choices to the specific properties of a problem in order to achieve optimal performance. Current methods for making such choices over available algorithms are often limited in applicability by the hardware on which the algorithms were benchmarked, the resource limits imposed on runs, and subject to bias caused by performance fluctuations in randomized algorithms. In many cases, these issues are caused by reliance on brittle performance measures, limiting practical application in academia and industry. This project aims to address these limitations in three ways. First, it will define a notion of robustness to guide algorithm selection, and identify properties of algorithms, experimental setups, and computational environments that affect robustness. Second, it will develop specific performance measures informed by this definition of robustness, and which are portable across different hardware platforms. Third, it will mitigate the impact of brittle performance measures through new approaches to building performance models based on machine learning. The project will result in the dissemination of shared data and benchmarks to the broader AI community, for example through the Algorithm Selection Library (ASlib).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

算法在现代社会中无处不在，并且是经济不可或缺的。无论是找到针对飞机，卡车和船只的包装和操作员的最佳分配，还是在不同语言之间翻译，解决的问题每天都变得更大，更具挑战性。至关重要的是，这是人工智能的进步。解决相同类型的问题通常有不同的方法，它们通常是协同的 - 在失败的情况下，另一个方法表现良好。 该项目中的AI技术允许自动选择给定问题的最佳方法。 这项研究将允许即使在困难的情况下也可以更坚固地做出此类选择，从而提高了性能并减少在实用系统中部署AI的努力。最终，该项目将使人类更容易开发高性能AI系统。Algorithm选择是将协同算法选择与问题的特定属性匹配的过程，以实现最佳性能。 在可用算法上做出此类选择的当前方法通常受到算法的标准标准算法的硬件，对运行施加的资源限制的限制，并受到随机算法中性能波动引起的偏见。 在许多情况下，这些问题是由于依赖脆性绩效指标而引起的，从而限制了学术界和行业中的实际应用。 该项目旨在通过三种方式解决这些限制。首先，它将定义一个鲁棒性的概念，以指导算法选择，并确定影响鲁棒性的算法，实验设置和计算环境的属性。其次，它将制定以这种鲁棒性定义为导致的特定绩效指标，它们在不同的硬件平台上可移植。第三，它将通过基于机器学习来构建性能模型的新方法来减轻脆性绩效指标的影响。 该项目将导致将共享数据和基准传播给更广泛的AI社区，例如，通过算法选择库（Aslib）（Aslib）。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子优点和更广泛的审查标准通过评估来进行评估的。

项目成果

Opening the Black Box: Automated Software Analysis for Algorithm Selection

• 发表时间: 2022
• 作者: Damir Pulatov;Marie Anastacio;Lars Kotthoff;H. Hoos

Transfer Learning for Performance Modeling of Deep Neural Network Systems

用于深度神经网络系统性能建模的迁移学习

• 发表时间: 2019
• 期刊: USENIX Conference on Operational Machine Learning
• 作者: Iqbal, Md Shariar;Kotthoff, Lars;Jamshidi, Pooyan
• 通讯作者: Jamshidi, Pooyan

mlr3pipelines - Flexible Machine Learning Pipelines in R

mlr3pipelines - R 中灵活的机器学习管道

• 发表时间: 2021
• 期刊: Journal of machine learning research
• 影响因子: 6
• 作者: Binder, Martin;Pfisterer, Florian;Lang, Michel;Schneider, Lennart;Kotthoff, Lars;Bischl, Bernd
• 通讯作者: Bischl, Bernd

Automated Benchmark-Driven Design and Explanation of Hyperparameter Optimizers

• DOI: 10.1109/tevc.2022.3211336
• 发表时间: 2021-11
• 期刊: IEEE Transactions on Evolutionary Computation
• 影响因子: 14.3
• 作者: Julia Moosbauer;Martin Binder;Lennart Schneider;Florian Pfisterer;Marc Becker;Michel Lang;Lars Kotthoff;Bernd Bischl

Is Algorithm Selection Worth It? Comparing Selecting Single Algorithms and Parallel Execution

算法选择值得吗？

• 发表时间: 2021
• 期刊: Proceedings of Machine Learning Research
• 作者: Kashgarani, Haniye;Kotthoff, Lars
• 通讯作者: Kotthoff, Lars