SI2-SSI: Integrating the NIMBLE Statistical Algorithm Platform with Advanced Computational Tools and Analysis Workflows

SI2-SSI：将 NIMBLE 统计算法平台与高级计算工具和分析工作流程集成

• 批准号: 1550488
• 负责人: Perry de Valpine
• 金额: $ 99.97万
• 依托单位: University of California-Berkeley
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-10-01 至 2022-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1550488&HistoricalAwards=false
• 关键词: SI2; SSI; Integrating; NIMBLE; Statistical

The software developed in this project will enable scientists to learn more from complex data and to share new analysis methods more easily. Increasingly, scientists in many fields aim to draw sound conclusions from large and complex data sets. Such fields include environmental biology, political science, education research, atmospheric and oceanic science, climate science, and many others. Data may be complex because many related variables are measured and/or because some measurements are not independent from each other. Non-independence can arise when some variables are measured repeatedly through time; or when measurements are made at nearby locations; or when measurements are made on groups of related individuals; or for a combination of those and other similar reasons. For such cases, general statistical methods have been developed to allow researchers to tailor their analysis to each data set in order to account for the relationships among the data. Such methods rely on computer algorithms to explore the range of possible conclusions given the uncertainties inherent in limited data. Within those general methods there are many varieties of specific approaches that have been and continue to be developed. Thus, a major software gap has emerged: Many new and evolving methods are not easily available for application by a wide range of scientists because there has not been a software framework that makes them easy to program and disseminate. This project will support continued development of the NIMBLE software to help fill that gap. As a result, scientists will be able to use computational analysis methods more flexibly, to combine and compare different algorithms more easily, to integrate such algorithms into other software workflows, and to gain better computational performance. This will enable more advanced and more routine use of some modern computational methods for analyzing complex data.The existing NIMBLE framework for hierarchical statistical models and algorithms comprises a model specification language, a language for programming model-generic algorithms within the R statistical environment, and a compiler that generates, compiles and interfaces to model- and algorithm-specific C++ for efficient execution. These enable general implementation and dissemination of methods such as Markov chain Monte Carlo, sequential Monte Carlo, and many related methods. In this project NIMBLE will be extended and generalized to be more powerful and flexible, enabling use in a variety of software workflows. Extensions to NIMBLE's core capabilities will include harnessing automatic differentiation and parallelization in generated C++, enhancements to its existing linear algebra capabilities, more efficient implementation of large statistical models including those with structural uncertainty such as latent group membership, and extensions to the statistical modeling language. Enhancements to facilitate integration of NIMBLE-generated models and algorithms with other software will include generation of stand-alone executables, generation of clearly defined application-programmer interfaces such as for use by Python, features to call user-provided libraries from algorithm code, features to load and save data via standard formats such as JSON and NetCDF, and separation of NIMBLE components into distinct packages. The project will include substantial outreach, training, and user community development. These activities will include development of uses cases in fields such as population and ecosystem ecology, oceanography, climate science, political science, and education. They will also include workshops, user meetings, key-user visits, and training material. This award by the Advanced Cyberinfrastructure Division is jointly supported by the NSF Directorate for Mathematical and Physical Sciences (Division of Mathematical Sciences).

该项目开发的软件将使科学家能够从复杂的数据中了解更多信息，并更轻松地共享新的分析方法。 许多领域的科学家越来越多地致力于从庞大而复杂的数据集中得出合理的结论。 这些领域包括环境生物学、政治学、教育研究、大气和海洋科学、气候科学等等。 数据可能很复杂，因为测量了许多相关变量和/或因为一些测量结果彼此不独立。 当某些变量随时间重复测量时，就会出现非独立性；或在附近地点进行测量时；或对相关个体进行测量时；或者出于这些原因和其他类似原因的组合。 对于这种情况，已经开发了通用统计方法，使研究人员能够针对每个数据集进行分析，以解释数据之间的关系。 鉴于有限数据固有的不确定性，此类方法依靠计算机算法来探索可能结论的范围。在这些一般方法中，有许多种类的具体方法已经并且仍在开发中。 因此，出现了一个重大的软件差距：许多新的和不断发展的方法不容易被广泛的科学家应用，因为没有一个软件框架可以使它们易于编程和传播。 该项目将支持 NIMBLE 软件的持续开发，以帮助填补这一空白。 从而，科学家将能够更灵活地使用计算分析方法，更轻松地组合和比较不同的算法，将这些算法集成到其他软件工作流程中，并获得更好的计算性能。 这将使一些现代计算方法能够更先进、更常规地用于分析复杂数据。现有的用于分层统计模型和算法的 NIMBLE 框架包括模型规范语言、R 统计环境中用于编程模型通用算法的语言，以及一个编译器，用于生成、编译特定于模型和算法的 C++ 并与其连接以实现高效执行。 这些使得马尔可夫链蒙特卡罗、顺序蒙特卡罗和许多相关方法等方法的通用实现和传播成为可能。在这个项目中，NIMBLE 将被扩展和通用化，变得更加强大和灵活，从而能够在各种软件工作流程中使用。 NIMBLE 核心功能的扩展将包括利用生成的 C++ 中的自动微分和并行化、增强其现有的线性代数功能、更有效地实现大型统计模型（包括具有结构不确定性的模型，例如潜在组成员身份）以及统计建模语言的扩展。 促进 NIMBLE 生成的模型和算法与其他软件集成的增强功能将包括生成独立的可执行文件、生成明确定义的应用程序程序员接口（例如供 Python 使用）、从算法代码调用用户提供的库的功能、功能通过 JSON 和 NetCDF 等标准格式加载和保存数据，并将 NIMBLE 组件分离到不同的包中。该项目将包括大量的外展、培训和用户社区发展。 这些活动将包括人口和生态系统生态学、海洋学、气候科学、政治学和教育等领域的用例开发。其中还将包括研讨会、用户会议、关键用户访问和培训材料。该奖项由先进网络基础设施部门颁发，并得到 NSF 数学和物理科学理事会（数学科学部门）的共同支持。