Enhancing Ensemble Diversity in Neural Ensemble Search for Uncertainty Quantification

增强神经集成搜索中的集成多样性以实现不确定性量化

• 批准号: 2872703
• 金额: --
• 依托单位: University of Oxford
• 依托单位国家: 英国
• 项目类别: Studentship
• 财政年份: 2023
• 资助国家: 英国
• 起止时间: 2023 至 无数据
• 项目状态: 未结题
• 来源: https://gtr.ukri.org/projects?ref=studentship-2872703
• 关键词: Enhancing; Ensemble; Diversity; Neural; Search

This project falls within the EPSRC `Artificial intelligence technologies' research area.It is an investigation into how to enhance uncertainty quantification through ensembles of deep neural networks.Especially for safety-critical applications such as autonomous driving or medical diagnosis, uncertainty quantification is relevant, however, neural networks are often badly calibrated and display either overly high or low confidence in their predictions. A popular non-Bayesian approach to improve calibration in neural networks is deep ensembles which average predictions of neural networks that have been trained from different random initialisations thereby achieving competitive predictive accuracy and calibration.To further enhance performance, methods proposed in the literature for automatically constructing ensembles of neural networks with varying architectures. They show that architectural variation leads to higher ensemble diversity resulting in ensembles with higher uncertainty calibration and robustness compared to the previous state-of-the-art deep ensembles.To automatically choose base learner architectures they suggest random search in combination with a greedy selection algorithm (NES-RS), an approach that is easily parallelisable, and a more sophisticated evolutionary algorithm based on regularized evolution (NES-RE), which exhibits better performance. Concurrent to this, Wenzel et al. proposed hyper-deep ensembles which aggregate ensembles over different random initialisations stratified over multiple random hyperparameters. However, they keep the architecture of their base learners fixed. The authors show that this approach for inducing diversity into ensembles also leads to increased predictive accuracy and calibration compared to deep ensembles.Therefore, the question arises whether a more directed search for hyperparameters can benefit the construction of more accurate, well-calibrated, and robust ensembles of neural networks, and, moreover, whether it is possible to achieve even better results by integrating the two approaches and extend NES-RE to search for optimal combinations of architecture and hyperparameters (e.g., dropout rate and different L2-regularizers) in base learners.This project will explore this novel approach by experimenting with different modifications to NES-RS and NES-RE that enable searching for ensembles with both varying architectures and hyperparameters simultaneously.Aims and Objectives1. Improve upon the current state-of-the-art NES-RE in terms of uncertainty quantification and robustness.2. If this is the case, investigate whether this approach to uncertainty quantification can aid safety in autonomous driving

该项目属于 EPSRC“人工智能技术”研究领域。它是对如何通过深度神经网络集合增强不确定性量化的研究。特别是对于自动驾驶或医疗诊断等安全关键应用，不确定性量化是相关的，然而，神经网络通常校准不当，并且对其预测表现出过高或过低的置信度。改进神经网络校准的一种流行的非贝叶斯方法是深度集成，它对从不同随机初始化训练的神经网络进行平均预测，从而实现有竞争力的预测精度和校准。为了进一步提高性能，文献中提出了自动构建的方法具有不同架构的神经网络集合。他们表明，与之前最先进的深度集成相比，架构变化会导致更高的集成多样性，从而使集成具有更高的不确定性校准和鲁棒性。为了自动选择基础学习器架构，他们建议结合贪婪选择算法进行随机搜索（NES-RS），一种易于并行化的方法，以及一种基于正则化进化（NES-RE）的更复杂的进化算法，它表现出更好的性能。与此同时，Wenzel 等人。提出了超深度集成，该集成在多个随机超参数上分层的不同随机初始化上聚合集成。然而，他们保持基础学习器的架构固定。作者表明，与深度集成相比，这种将多样性引入集成的方法还可以提高预测准确性和校准能力。因此，出现的问题是，更直接地搜索超参数是否有利于构建更准确、校准良好和鲁棒的模型神经网络的集成，此外，是否有可能通过整合这两种方法并扩展 NES-RE 来搜索架构和超参数（例如，dropout 率）的最佳组合来获得更好的结果和不同的 L2 正则化器）。该项目将通过试验对 NES-RS 和 NES-RE 的不同修改来探索这种新颖的方法，从而能够同时搜索具有不同架构和超参数的集成。目的和目标 1。在不确定性量化和鲁棒性方面改进了当前最先进的NES-RE。2．如果是这种情况，请调查这种不确定性量化方法是否有助于自动驾驶的安全