Elements: Basil: A Tool for Semi-Automatic Containerization, Deployment, and Execution of Scientific Applications on Cloud Computing and Supercomputing Platforms

要素：Basil：在云计算和超级计算平台上半自动容器化、部署和执行科学应用程序的工具

• 批准号: 2314203
• 负责人: Ritu Arora
• 金额: $ 57.08万
• 依托单位: Wayne State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-01-01 至 2025-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2314203&HistoricalAwards=false
• 关键词: Elements; Basil; Tool; Semi; Automatic

The "containerization" of software applications future-proofs them, helps in their long-term preservation, makes them portable across different hardware platforms, ensures reproducible results, and makes them convenient to disseminate. Docker and Singularity are two popular software technologies for containerizing scientific applications and are widely supported on different hardware platforms. However, their adoption involves a steep learning curve, especially when it comes to developing secure and optimized images of the applications of interest. A large number of domain-scientists and scholars are usually not formally trained at containerizing their applications with Docker and Singularity, and spend a significant amount of their time in porting their applications to different cloud computing and supercomputing platforms. The process of porting applications having multiple software dependencies and sensitivities to specific software versions can be especially arduous for such users. To assist them, this project is developing BASIL - a tool for semi-automatically containerizing the scientific applications, frameworks, and workflows. This project will deliver BASIL through a web portal, as a command-line tool, and through APIs. BASIL has a broad applicability across multiple domains of deep societal impact such as artificial intelligence, drug discovery, and earthquake engineering. By enabling the preservation of valuable legacy software and making them usable for several years in future, BASIL will save cost and time in software rewriting and software installations, and thus contribute towards advancing the prosperity of the society. The project will result in educational content on “Introduction to Containerization” and students engaged in the project will develop valuable skills in the areas of national interest such as supercomputing/High Performance Computing (HPC) and cloud computing. BASIL will be the first tool of its kind that can semi-automatically generate secure, optimized, and trustworthy container images with clear information on how to use the images under appropriate licenses. The rules for optimizing the images will be derived from expert knowledge and best practices, such as multi-stage builds and reordering the sequencing of commands to take advantage of caching so that the overall time involved in building the images is reduced. Users of the BASIL tool will provide the recipes for building their applications/workflows in one of the following forms (1) Makefiles/CMakefiles, (2) scripts, (3) commands, or (4) a text-file with predefined keywords and notations using templates provided by the project team. These recipes will be parsed, and Dockerfiles or Singularity definition files will be generated. The parser developed in this project will be another novel contribution of the project. Using a generated Dockerfile or Singularity definition file, a Docker or Singularity image will be built. Next, the image will be scanned for any vulnerabilities, signed, and if the user desires, released in public registries with appropriate licenses. These container images can be tested using the BASIL web portal, and can be pulled to run or deploy on diverse hardware platforms. This award by the Office of Advanced Cyberinfrastructure is jointly supported by the Physics at the Information Frontier in the Division of Physics within the Directorate for Mathematical and Physical Sciences.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

软件应用程序的“容器化”将来可以防止它们，可以帮助他们长期准备，使它们可以在不同的硬件平台上移植，确保可重现的结果，并使它们方便地进行传播。 Docker和Singularity是两种流行的软件技术，用于集装科学应用程序，并在不同的硬件平台上得到了广泛的支持。但是，它们的采用涉及钢曲线，尤其是在开发感兴趣应用的安全和优化图像时。通常，许多领域科学家和学者通常不会对使用Docker和Singularity的应用程序进行正式培训，并花费大量时间将其应用程序移植到不同的云计算和超级计算平台上。对于此类用户而言，移植具有多个软件依赖性和对特定软件版本的敏感性的应用程序的过程可能特别艰巨。为了帮助他们，该项目正在开发罗勒 - 一种半自动的工具，其中包含科学应用，框架和工作流程。该项目将通过Web门户，命令行工具和API传递罗勒。罗勒在人工智能，药物发现和地震工程等深层社会影响的多个领域中具有广泛的适用性。通过启用有价值的旧软件并使其在未来几年内可用，罗勒将节省软件重写和软件安装的成本和时间，从而有助于促进社会的繁荣。该项目将导致有关“容器化简介”的教育内容，从事该项目的学生将在国家兴趣领域（例如超级计算/高性能计算（HPC）和云计算）中发展有价值的技能。罗勒将是同类工具，可以半自动地生成安全，优化和可信赖的容器图像，并具有有关如何在适当许可下使用图像的明确信息。优化图像的规则将源自专家知识和最佳实践，例如多阶段构建和重新排序命令的测序以利用缓存，以便减少构建图像的总体时间。罗勒工具的用户将提供以下表格之一（1）makefiles/cmakefiles，（2）脚本，（3）命令，或（4）使用项目团队提供的模板提供预定义的关键字和符号的文本文件。这些配方将被解析，将生成Dockerfiles或Singularity定义文件。该项目中开发的解析器将是该项目的另一个新颖贡献。使用生成的dockerfile或奇异定义文件，docker或奇异性图像接下来，将扫描图像以符合任何漏洞，并签名，如果用户愿望在公共注册表中发布了具有适当许可的公共注册表。这些容器图像可以使用Basil Web门户进行测试，并且可以拉出以在潜水员硬件平台上运行或部署。高级网络基础设施办公室的奖项由物理学在数学和物理科学局内物理部门的物理领域共同支持。该奖项反映了NSF的法定任务，并通过基金会的知识分子和更广泛的影响审查审查标准来通过评估来诚实地诚实地支持。