Collaborative Research: Elements: FaaSr: Enabling Cloud-native Event-driven Function-as-a-Service Computing Workflows in R

协作研究：要素：FaaSr：在 R 中启用云原生事件驱动的函数即服务计算工作流程

• 批准号: 2311123
• 负责人: Renato Figueiredo
• 金额: $ 50万
• 依托单位: University of Florida
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-09-15 至 2026-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2311123&HistoricalAwards=false
• 关键词: Collaborative; Research; Elements; FaaSr; Enabling

This project develops FaaSr, a new software that will facilitate the programming and deployment of scientific computing applications written in the R language in Function-as-a-Service (FaaS) cloud computing infrastructures. The FaaS model of cloud computing supports dynamic, on-demand execution of computing functions in servers that are automatically provisioned and managed, in a way that is both cost-effective and scalable: users do not need to manage cloud servers (including on-demand scaling) nor pay for idle time of unutilized servers. The FaaS model thus has much potential for reducing the complexity and cost of performing scientific computing in cloud infrastructures. To date, however, FaaS platforms have been primarily designed to support Web-based applications, resulting in a major gap between existing FaaS platforms and the scientific community. This gap is particularly evident in the environmental sciences, where R is the focal programming language. This is because: 1) there is no native support for the R language in FaaS platforms, and 2) each FaaS platform has a unique interface to deploy and manage workflows consisting of multiple functions, thereby creating barriers for users to develop and deploy applications on one or more FaaS platforms. This project bridges this gap by developing open-source software to accelerate the adoption of event-driven FaaS workflows for scientific applications. The FaaSr software will be distributed as an easy-to-install R package and will provide simple interfaces to programmers, while supporting multiple open-source and commercial cloud computing infrastructures. The software will support a wide range of scientific computing applications, in particular those that require dynamic event-driven processing (such as forecasting and continuous data quality) in environmental science subfields (including ecology and biodiversity). Ultimately, the project aims to develop scalable, generalizable, and robust workflows that will advance the capacity, practice, and training opportunities for ecological forecasting, an active area of scientific research poised to significantly increase predictive capacity for effective environmental decision-making and management. The FaaSr software developed in this project will greatly expand the adoption of FaaS cloud computing infrastructure. Currently, there are significant challenges to be overcome before scientific applications written in the R language can fully realize the potential of FaaS platforms, because R is not supported natively, and because different platforms have different, incompatible programming interfaces. Furthermore, scientific applications require workflows consisting of multiple functions that are executed dynamically and communicate by exchanging data as files in cloud storage. Different FaaS platforms have different programming interfaces to accomplish these capabilities, leading to increased complexity for developers and users. This collaborative, interdisciplinary project overcomes these challenges by integrating expertise in distributed systems, ecology, and forecasting together to design and implement software that: is driven by scientific computing use cases; creates easy-to-use interfaces; and builds on state-of-the-art distributed computing techniques and frameworks. Specifically, the FaaSr software will make multiple novel technical contributions, including: 1) it will allow end users to program a workflow at a high abstraction level and with the R language; 2) it will include a unified, easy-to-use interface for handling event invocation and argument parsing that hides the complexity of programming for multiple FaaS interfaces from developers, while supporting multiple FaaS frameworks, including GitHub Actions, OpenWhisk, IBM Cloud Functions, and Amazon Web Services Lambda; 3) it will include an easy-to-use interface for handling cloud data storage and access that hides low-level details (e.g., access endpoints and credentials) using de-facto standard interfaces and file formats; and 4) it will implement a unified approach to compose directed acyclic graph workflows that can be automatically mapped to programming interfaces supported by different FaaS platforms. Experiences with the design, implementation, and deployment of FaaSr will contribute new techniques and technologies in distributed/cloud computing, with lakes and reservoirs studied as part of this project providing a realistic testbed for assessing performance, extensibility, and availability of the software. Furthermore, the team will build on and expand its existing program for cross-disciplinary research exchanges of undergraduate and graduate students that provide novel training at the intersection of computer science, freshwater science, and ecosystem modeling.This award by the NSF Office of Advanced Cyberinfrastructure is jointly supported by the NSF Directorate for Biological Sciences.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目开发了FAASR，这是一种新软件，将促进用r语言编写的科学计算应用程序的编程和部署，该应用程序在功能上为AS-AS-AS-Service（FAAS）云计算基础架构。云计算的FAA模型支持动态，按需执行计算功能的服务器中自动配置和管理的服务器，既具有成本效益又可扩展的方式：用户不需要管理云服务器（包括按需缩放），也不需要用于未充分利用服务器的空闲时间。因此，FAAS模型具有降低云基础架构进行科学计算的复杂性和成本的很大潜力。但是，迄今为止，FAAS平台主要旨在支持基于Web的应用程序，从而导致现有的FAAS平台与科学界之间存在重大差距。在环境科学中尤其明显，其中R是焦点编程语言。这是因为：1）在FAAS平台中没有对R语言的本机支持，而2）每个FAAS平台都有一个独特的接口来部署和管理由多个功能组成的工作流程，从而为用户创建了在一个或多个FAAS平台上开发和部署应用程序的障碍。该项目通过开发开源软件来弥合这一差距，以加速采用事件驱动的FAAS工作流程以进行科学应用。 FAASR软件将作为易于启动的R软件包分发，并将为程序员提供简单的接口，同时支持多个开源和商业云计算基础结构。该软件将支持广泛的科学计算应用程序，特别是那些需要动态事件驱动的处理（例如环境科学子场（包括生态学和生物多样性））的动态驱动处理（例如预测和连续数据质量）。最终，该项目旨在开发可扩展，可推广和强大的工作流程，以促进生态预测的能力，实践和培训机会，这是一个有效的科学研究领域，该领域有望显着提高有效的环境决策和管理的预测能力。该项目开发的FAASR软件将大大扩展FAAS云计算基础架构的采用。当前，在用R语言编写的科学应用程序可以完全意识到FAAS平台的潜力之前，要克服巨大的挑战，因为R在本地不受支持，并且由于不同的平台具有不同的，不兼容的编程接口。此外，科学应用程序需要工作流，这些工作流由多个功能组成，这些功能由动态执行并通过将数据作为云存储中的文件交换来进行通信。不同的FAA平台具有不同的编程接口来完成这些功能，从而增加了开发人员和用户的复杂性。这个合作的跨学科项目通过将分布式系统，生态学和预测合并在一起的专业知识来设计和实施：由科学计算用例驱动的软件来克服这些挑战；创建易于使用的接口；并建立在最先进的分布式计算技术和框架的基础上。具体而言，FAASR软件将做出多种新颖的技术贡献，包括：1）它将允许最终用户以高抽象级别和R语言进行编程； 2）它将包括一个统一的，易于使用的界面，用于处理事件调用和参数解析，该界面隐藏了开发人员的多个FAA界面编程的复杂性，同时支持多个FAAS框架，包括GitHub Action，OpenWhisk，OpenWhisk，IBM Cloud Cloud功能和Amazon Web Services Lambda lambda lambda; 3）它将包含一个易于使用的接口，用于处理云数据存储和访问，该接口使用deFacto标准接口和文件格式隐藏了低级详细信息（例如，访问端点和凭据）； 4）它将实施一种统一的方法来组成有向的无环图工作流，该方法可以自动映射到不同FAAS平台支持的编程接口。 FAASR的设计，实施和部署的经验将在分布式/云计算中贡献新的技术和技术，作为该项目的一部分，湖泊和水库研究了湖泊和水库，为评估性能，可扩展性和可用性提供了现实的测试。此外，该团队将在本科生和研究生的跨学科研究交流和扩展其现有计划中，在计算机科学，淡水科学和生态系统建模的交集中提供新颖的培训。NSF获得了高级Cyber​​infrasture的NSF奖。通过使用基金会的知识分子和更广泛影响的评论标准来通过评估来支持。