Frameworks: Re-Engineering Galaxy for Performance, Scalability and Energy Efficiency

框架：重新设计 Galaxy 以提高性能、可扩展性和能源效率

• 批准号: 1931531
• 负责人: Mahmut Kandemir
• 金额: $ 350万
• 依托单位: Pennsylvania State Univ University Park
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1931531&HistoricalAwards=false
• 关键词: Frameworks; Re; Engineering; Galaxy; Performance

Biomedical research is an important branch of science that deals with the problem of studying biological processes and identifying, preventing and curing diseases. This research forms the pathway to the discovery of new medicines as well as new therapies. As such, biomedical research is crucial to advance the national health and prosperity. Given the geographically distributed research groups and biomedical labs, collaborative science plays a very important role in biomedical research. Galaxy is an open source, web-based framework that is extensively used by more than 20,000 researchers world-wide for conducting research in many application domains, the most prominent of which is biomedical research. It provides a web-based environment using which scientists perform various computational analyses on their data, exchange results from these analyses, explore new research concepts, facilitate student training, and preserve their results for future use. Galaxy currently runs on a large variety of high-performance computing (HPC) platforms including local clusters, supercomputers in national labs, public datacenters and Cloud. Unfortunately, while most of these systems supplement conventional CPUs with significant accelerator capabilities (in the form of Graphical Processing Units (GPUs) and/or Field-Programmable Gate Arrays (FPGAs)), the current Galaxy implementation does not take advantage of these powerful accelerators. This project enhances the Galaxy framework so that it can take full advantage of the tremendous computational capabilities offered by GPUs and FPGAs. By doing so, the important applications running under Galaxy experiences significant speedups, thereby accelerating scientific discoveries. This project consists of four complementary tasks, which follow a logistic progression as follows: Task-I focuses on redesigning existing Galaxy tools with GPU/FPGA support and integrate them to Galaxy tool-chains; Task-II provides containerization support for the tools and accelerator-aware orchestration for running Galaxy on cloud platforms; Task-III implements specific policy driven scheduling schemes for Task-I and Task-II; and finally, Task-IV redesigns Galaxy storage to speed up execution and reduce bottlenecks related to data transfer. The proposed enhancements to Galaxy enables the integration of innovation with discovery by providing a state-of-the art experimental platform to a larger community of researchers across several disciplines. On the broader impact and outreach/educational front, this project impacts the performance and energy efficiency of Galaxy tools and applications and improves the productivity of a typical Galaxy user tremendously; that is, the main beneficiaries of this project are thousands of members of existing Galaxy Community. However, this project also (i) helps existing GPU and FPGA based (non-Galaxy) applications start using Galaxy, thereby taking full advantage of all existing toolsets within the framework, (ii) enables Galaxy tools to take better advantage of emerging cluster scheduling capabilities, and (iii) creates a synergy with concurrent Galaxy related efforts and existing infrastructure efforts the PIs are involved with, to further expedite scientific discoveries. As such, this proposed system support will have a broad societal impact via the enhanced Galaxy system support. On the education side, the project involves under-represented groups in computer science as well as in bio-informatics, outreach to undergraduates, various K-12 related activities (Science-U, CSATS, VIEW), and engagement with researchers in other disciplines (e.g., natural language processing, image processing, drug discovery and cosmology) via a workshop open to the Galaxy community.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

生物医学研究是科学的一个重要分支，涉及研究生物过程以及识别，预防和治愈疾病的问题。这项研究构成了发现新药物以及新疗法的途径。因此，生物医学研究对于促进国家健康和繁荣至关重要。鉴于地理分布的研究小组和生物医学实验室，协作科学在生物医学研究中起着非常重要的作用。 Galaxy是一个开源的，基于网络的框架，全世界有20,000多名研究人员广泛使用了许多应用领域的研究，其中最杰出的是生物医学研究。 它提供了一个基于网络的环境，科学家对其数据进行了各种计算分析，从这些分析中进行交换结果，探索新的研究概念，促进学生培训并保留其结果以供将来使用。 Galaxy目前在各种高性能计算（HPC）平台上运行，包括本地群集，国家实验室中的超级计算机，公共数据中心和云。不幸的是，尽管这些系统中的大多数都以显着的加速器功能（以图形处理单元（GPU）和/或野外可编程的门阵列（FPGAS）的形式补充常规CPU，但当前的星系实现并不能利用这些强大的加速器。该项目增强了银河系框架，因此它可以充分利用GPU和FPGA提供的巨大计算功能。通过这样做，在银河系下运行的重要应用程序会经历大量的加速，从而加速了科学发现。 该项目由四个互补任务组成，遵循逻辑进度如下：任务-I着重于重新设计了现有的GALAXY工具，并将其集成到Galaxy Tool-tool-chins； II TASK-II为工具和加速器感知的编排提供了容器化支持，用于在云平台上运行Galaxy；任务III实施特定的策略驱动的调度计划，用于任务I和任务II；最后，任务IV重新设计了Galaxy存储以加快执行速度并减少与数据传输相关的瓶颈。拟议的对银河系的增强功能可以通过为在几个学科的一个更大的研究人员社区提供最先进的实验平台，从而使创新与发现结合。在更广泛的影响力和外展/教育方面，该项目影响了银河工具和应用的性能和能源效率，并极大地提高了典型的Galaxy用户的生产力；也就是说，该项目的主要受益者是现有星系社区的成千上万成员。 However, this project also (i) helps existing GPU and FPGA based (non-Galaxy) applications start using Galaxy, thereby taking full advantage of all existing toolsets within the framework, (ii) enables Galaxy tools to take better advantage of emerging cluster scheduling capabilities, and (iii) creates a synergy with concurrent Galaxy related efforts and existing infrastructure efforts the PIs are involved with, to further expedite scientific discoveries.因此，该提出的系统支持将通过增强的星系系统支持产生广泛的社会影响。 在教育方面，该项目涉及计算机科学以及生物信息学，对本科生的宣传，各种K-12相关活动（Science-U，CSAT，观点）以及与其他学科中的研究人员的互动（例如，通过自然语言处理，图像处理，药物发现和相关的宣传）与其他学科的研究人员的参与。使用基金会的知识分子优点和更广泛的审查标准，通过评估被认为值得支持。

项目成果

Kube-Knots: Resource Harvesting through Dynamic Container Orchestration in GPU-based Datacenters

• DOI: 10.1109/cluster.2019.8891040
• 发表时间: 2019-09
• 期刊: 2019 IEEE International Conference on Cluster Computing (CLUSTER)
• 作者: P. Thinakaran;Jashwant Raj Gunasekaran;Bikash Sharma;M. Kandemir;C. Das

Fifer: Tackling Resource Underutilization in the Serverless Era

• DOI: 10.1145/3423211.3425683
• 发表时间: 2020-12
• 期刊: Proceedings of the 21st International Middleware Conference
• 作者: Jashwant Raj Gunasekaran;P. Thinakaran;N. Nachiappan;M. Kandemir;C. Das

Compression Algorithm for Colored de Bruijn Graphs

彩色 de Bruijn 图的压缩算法

• 发表时间: 2023
• 期刊: 23rd International Workshop on Algorithms in Bioinformatics (WABI 2023
• 作者: Rahman, Amatur;Dufresne, Yoann;Medvedev, Paul
• 通讯作者: Medvedev, Paul

Multiverse: Dynamic VM Provisioning for Virtualized High Performance Computing Clusters

• DOI: 10.1109/ccgrid49817.2020.00-80
• 发表时间: 2020-05
• 期刊: 2020 20th IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing (CCGRID)
• 作者: Jashwant Raj Gunasekaran;Michael Cui;P. Thinakaran;Josh Simons;M. Kandemir;C. Das

Cocktail: A Multidimensional Optimization for Model Serving in Cloud

Cocktail：云中模型服务的多维优化

• 发表时间: 2023
• 期刊: 19th USENIX Symposium on Networked Systems Design and Implementation.
• 作者: Jashwant Raj Gunasekaran, Cyan Subhra