EAGER: Search-Accelerated Markov Chain Monte Carlo Algorithms for Bayesian Neural Networks and Trillion-Dimensional Problems

EAGER：贝叶斯神经网络和万亿维问题的搜索加速马尔可夫链蒙特卡罗算法

• 批准号: 2404989
• 负责人: Peter Behroozi
• 金额: $ 16.33万
• 依托单位: University of Arizona
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2024
• 资助国家: 美国
• 起止时间: 2024-04-01 至 2025-03-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2404989&HistoricalAwards=false
• 关键词: EAGER; Search; Accelerated; Markov; Chain

Machine learning with deep neural networks occurs in every scientific field. In astronomy, machine learning (ML) applications are wide-ranging. Advanced ML methods involve large numbers of tunable variables, in the range of millions to trillions of parameters. Current methods for tuning these parameters can find a single best-fitting model but are unable to produce the range of ML models that match observations. Knowing the range of uncertainties for a given method is of fundamental importance in science. More broadly, there are many areas beyond science where knowing the range of uncertainties is important (e.g., driverless cars, medical, and military applications). This proposal introduces a new algorithm that will explore uncertainties for trillion-dimensional models and beyond (encompassing the dimensionality of current large neural networks). For such trillion-dimensional parameter spaces, the method would be five hundred times faster than the best previous approaches. This method could revolutionize machine learning across scientific and commercial fields. The ability to show that all reasonable neural networks give similar results would directly address present problems of robustness and reproducibility, as well as rigorously quantify model uncertainties. This proposal also involves direct broader impact work with veterans transitioning from the military to college. This proposal would advance the frontiers of sampling algorithm performance, with typical O(log D) or better scaling with dimensionality, as well as provide new knowledge about the geometry of high-dimensional posterior distributions, including those for deep neural networks. Advancing sampling algorithm performance would allow a broad range of problems to be addressed in astronomy that would otherwise be computationally intractable, especially reconstruction problems (e.g., field-level inference for cosmology or blended source reconstruction) and high-dimensional modeling (e.g., modeling the joint distribution of physical properties of multiple galaxies simultaneously, given their luminosity, spatial, and redshift distributions). Advancing understanding of the geometry of high-dimensional posterior distributions would enable the development of more optimized algorithms for exploration and sampling, particularly for advanced neural networks, further reducing the barriers to robust posterior distributions. The results of this proposal will be released as an open-source implementation of the algorithm as well as an open-access accompanying paper.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

使用深度神经网络的机器学习存在于每个科学领域。在天文学中，机器学习 (ML) 应用非常广泛。 高级机器学习方法涉及大量可调变量，参数范围为数百万到数万亿。当前调整这些参数的方法可以找到单个最佳拟合模型，但无法生成与观察结果相匹配的 ML 模型范围。了解给定方法的不确定性范围对于科学至关重要。更广泛地说，在科学之外的许多领域，了解不确定性的范围很重要（例如，无人驾驶汽车、医疗和军事应用）。该提案引入了一种新算法，将探索万亿维模型及更高维度（包括当前大型神经网络的维度）的不确定性。对于如此万亿维的参数空间，该方法将比之前最好的方法快五百倍。这种方法可以彻底改变科学和商业领域的机器学习。能够证明所有合理的神经网络给出相似的结果将直接解决当前的鲁棒性和可重复性问题，并严格量化模型的不确定性。该提案还涉及对从军队过渡到大学的退伍军人进行直接更广泛影响的工作。该提案将通过典型的 O(log D) 或更好的维度缩放来推进采样算法性能的前沿，并提供有关高维后验分布几何的新知识，包括深度神经网络的几何知识。提高采样算法的性能将允许解决天文学中的广泛问题，否则这些问题在计算上会很困难，特别是重建问题（例如，宇宙学的场级推理或混合源重建）和高维建模（例如，对多个星系同时物理属性的联合分布（考虑到它们的光度、空间和红移分布）。增进对高维后验分布几何形状的理解将有助于开发更优化的探索和采样算法，特别是对于高级神经网络，进一步减少稳健后验分布的障碍。该提案的结果将作为算法的开源实现以及开放获取的随附论文发布。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响进行评估，被认为值得支持审查标准。