Development Of Advanced Computer Hardware And Software

先进计算机硬件和软件的开发

• 批准号: 10929241
• 负责人: Bernard R Brooks
• 金额: $ 74.52万
• 依托单位: NATIONAL HEART, LUNG, AND BLOOD INSTITUTE
• 依托单位国家: 美国
• 资助国家: 美国
• 起止时间: 至
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/10929241
• 关键词: Advanced Development; Affect; Algorithms; Amber; Architecture; Area; Biophysics; Bite; Carbohydrates; Cluster Analysis; Code; Collaborations; Communication; Computational Biology; Computer Hardware; Computer software; Computers; Computing Methodologies; Consumption; Coupled; Data Collection; Derivation procedure; Development; Dimensions; Electrostatics; Entropy; Evaluation; Formulation; Future; Generations; High Performance Computing; Hydrogen; Individual; Infrastructure; Internet; Intervention; Investments; Lead; Length; Link; Lipids; Measures; Memory; Methods; Molecular Biology; Molecular Conformation; Molecular Structure; Performance; Periodicals; Personal Computers; Positioning Attribute; Preparation; Process; Property; Residual state; Resources; Rotation; Running; SARS-CoV-2 spike protein; Software Tools; Speed; Stress; Structure; Surface; Surface Tension; System; Techniques; Technology; Time; United States National Aeronautics and Space Administration; Update; Water; Work; Workload; comparative; computer grid; computerized tools; cost; cost effective; data exchange; data mining; design; improved; indexing; large datasets; molecular dynamics; molecular scale; operation; parallel computer; parallel processing; parallelization; pressure; programs; scientific computing; simulation; software development; solute; theories; tool; web-based tool; Advanced Development; Affect; Algorithms; Amber; Architecture; Area; Biophysics; Bite; Carbohydrates; Cluster Analysis; Code; Collaborations; Communication; Computational Biology; Computer Hardware; Computer software; Computers; Computing Methodologies; Consumption; Coupled; Data Collection; Derivation procedure; Development; Dimensions; Electrostatics; Entropy; Evaluation; Formulation; Future; Generations; High Performance Computing; Hydrogen; Individual; Infrastructure; Internet; Intervention; Investments; Lead; Length; Link; Lipids; Measures; Memory; Methods; Molecular Biology; Molecular Conformation; Molecular Structure; Performance; Periodicals; Personal Computers; Positioning Attribute; Preparation; Process; Property; Residual state; Resources; Rotation; Running; SARS-CoV-2 spike protein; Software Tools; Speed; Stress; Structure; Surface; Surface Tension; System; Techniques; Technology; Time; United States National Aeronautics and Space Administration; Update; Water; Work; Workload; comparative; computer grid; computerized tools; cost; cost effective; data exchange; data mining; design; improved; indexing; large datasets; molecular dynamics; molecular scale; operation; parallel computer; parallel processing; parallelization; pressure; programs; scientific computing; simulation; software development; solute; theories; tool; web-based tool

Constant pressure simulation on GPUs Most molecular dynamics packages do not handle the non-pairwise contributions to the virial, resulting in non-rigorous isobaric ensemble simulations. Through the derivation of these contributions and its implementation via the Langevin Piston algorithm in our apoCHARMM code, constant pressure (isobaric ensemble) as well as constant surface tension, constant surface area and related ensembles can now be simulated efficiently. Mixed precision iterative refinement solution for induced dipoles using tensor cores Solving the induced dipoles in a self-consistent fashion is one of the most challenging aspects of polarizable force fields. Since it is the most time consuming component, we have devised an iterative mixed precision solution for this calculation using the hardware units on the GPUs called Tensor Cores. These special units allow GEMM operations using only 16-bit precision matrices. However, they provide high performance. While FP64 throughput is limited to 9.7 TFlops, tensor cores provide up to 312 FP16 TFlops. By iteratively refining the residual, we improve the accuracy up to 64-bit precision. Overall we are able to speed up the induced polarization calculation by 3-4X. This work will make polarizable force field simulations more accessible on the GPUs. P21 reciprocal space calculation Reciprocal space calculation in P21 is challenging because the classical Ewald formulation works with only translational symmetry and does not support rotational symmetry.We have developed the Ewald formulation for long-range electrostatics for systems under P21 periodic boundary condition. By expressing the contribution from the rotated asymmetric unit in terms of the primary asymmetric unit, we were able to express the reciprocal space potential only in terms of the latter. This reduces the computational work by half. P21 periodic boundary conditions are important for balancing the stress disequilibrium between the layers of the lipids during the simulation. Quantifying the Effects of Lossy Compression on Energies Calculated from Molecular Dynamics Trajectories MD simulations can now be run for increasingly longer lengths of time and on larger systems (> 100,000 atoms). There is a need to store these trajectories in as efficient a way as possible without sacrificing too much precision. We have explored how quantization and compression affects the precision of not only atomic positions (as is typically done), but also the energies calculated from such trajectories, and have compared to a wide variety of new and existing trajectory formats (21 total). We found that while many geometric properties (distances, RMSD, RDFs, etc.) can be reproduced from existing compressed trajectory formats with low precision (like XTC, 0.01 ), bond energies involving hydrogen are particularly sensitive to precision loss. As a result, we have developed a quantization-based compression new format that compresses to about 66% of the size of the original NetCDF trajectory, has a positional accuracy of 5x10-5 , has an energy root-mean-square error of less than 0.1 kcal/mol, and is almost as fast to read as the original uncompressed trajectory. Adding Automatic Parameter Downloads to a Software Tool for Fast PDB-to-Parameter Generation for Molecular Dynamics Simulations Setting up molecular dynamics simulations from experimentally-determined structures is often complicated by a variety of factors, particularly when the structure to be simulated contains carbohydrates (e.g. the SARS-CoV-2 spike protein), since these have several forms and be linked in a variety of ways. Previously, we developed a stand-alone tool called prepareforleap, implemented in the widely used and freely-available software CPPTRAJ (which now has close to 4k citations) that facilitates the preparation of structures for molecular dynamics simulations with the Amber Biomolecular simulation package. This software tool is a stand-alone program that requires no internet access and little-to-no user intervention, which differentiates it from existing web-based tools. Addition of Clustering via Extended Similarity Metrics to CPPTRAJ Cluster analysis is data-mining technique that can be applied to a collection of data points to create groups of points according to some measure of similarity. In the context of MD simulations, this typically means identifying important and unique clusters of conformations from trajectories that typically contain thousands to sometimes millions of structures. As such, cluster analysis is a very important tool in the analysis of MD simulations for guiding future analysis by reducing the dimensionality of extremely large data sets. The calculation of pairwise distances between any two points, which can be a bottleneck and scales poorly (O(n2)). Miranda-Quintana et al. recently introduced a new clustering method based on determining extended similarity indices instead of pairwise distances, which reduces the scaling to O(n). In collaboration with them, we are now implementing this new clustering method into CPPTRAJ. This will enable comparatively rapid cluster analysis of extremely large data sets. Parallelization of Grid Inhomogeneous Solvation Theory Calculations The enthalpic portion of the GIST method requires the calculation of water-water and water-solute energy on the grid; this is usually the most time-consuming part of GIST. We are now working to increase the speed of this calculation by parallelizing it with MPI, dividing up the incoming trajectory frames among multiple processors. This is particularly attractive because it requires very little communication between individual processes during trajectory processing, meaning the calculation should scale well to large processor counts. In addition to the energy calculation, we are also parallelizing the entropy calculation via MPI, which must happen after trajectory processing since it requires information from all trajectory frames. This parallelized GIST method is being developed and will be freely available in the CPPTRAJ analysis software. GPU-parallelization of Time-consuming Calculations in CPPTRAJ We are porting several methods to run simultaneously on multiple GPUs. Acquisitions and Hardware Upgrades: Nvidia A100 Systems: Over the past year, strategic investments have been made in our infrastructure. We acquired several systems, specifically designed to enhance our high-performance computing capabilities. Among these are two 8-way Nvidia A100 compute nodes and one 4-way Nvidia A100 compute node. These systems are equipped with multiple GPUs connected via NVLink, Nvidia's cutting-edge GPU-to-GPU interconnect technology. Nvidia H100 System: As a testament to our commitment to staying abreast with the latest in computational technology, we procured an 8-way Nvidia H100 system in August 2023. The rationale behind these acquisitions is twofold: Scalability: The availability of systems with multiple GPUs, especially when interconnected via NVLink, offers an unparalleled opportunity for our team. With these systems, we are not only gearing up for current workloads but also supporting our development of advanced codebases. Specialization: Molecular biology, especially when approached through computational methods, demands high granularity and computational power. The Nvidia A100 and H100 systems are the best way for us to meet our large computational needs. They promise enhanced parallel processing capabilities, faster memory access, and improved data transfer rates, all of which are vital for large-scale molecular simulations and computations. In conclusion, our investments over the past year have been methodical, targeting both current and future demands of computational biology and biophysics.

GPU上的恒定压力模拟 大多数分子动力学套件都无法处理对病毒的非对贡献，从而产生了非鲁孔的同质集合模拟。通过在我们的Apocharmm代码中通过Langevin活塞算法实现这些贡献及其实施，可以有效地模拟恒定压力（等距集合）以及恒定的表面张力，恒定的表面积和相关组件。 使用张量芯的混合精度迭代精制溶液用于诱导偶极子 以自洽的方式解决诱导的偶极子是可极化力场最具挑战性的方面之一。由于它是最耗时的组件，因此我们使用称为张量核心的GPU上的硬件单元为此计算设计了一个迭代混合精度解决方案。这些特殊单元仅允许仅使用16位精度矩阵操作。但是，它们提供了高性能。虽然FP64吞吐量限制为9.7 TFLOPS，但张量芯可提供高达312 FP16 TFLOPS。通过迭代完善残差，我们提高了高达64位精度的准确性。总体而言，我们能够将诱导的极化计算加快3-4倍。这项工作将使可极化的力场模拟在GPU上更容易访问。 P21相互空间计算 p21中的相互空间计算是具有挑战性的，因为经典的ewald配方仅与翻译对称性一起起作用，并且不支持旋转对称性。我们已经开发了在P21周期性边界条件下针对系统的远程静电的Ewald公式。通过表达旋转不对称单元的贡献，就主要的不对称单位而言，我们只能在后者方面表达相互的空间潜力。这将计算工作减少了一半。 p21周期性边界条件对于平衡模拟过程中脂质层之间的应力不平衡很重要。 量化有损压缩对从分子动力学轨迹计算的能量的影响 现在可以在越来越长的时间内和更大的系统（> 100,000个原子）上运行MD模拟。有必要以尽可能高效的方式存储这些轨迹，而不会牺牲过多的精度。我们已经探索了量化和压缩如何影响原子位置的精度（通常是这样做），还影响了从这些轨迹计算出的能量，并与多种新的和现有的轨迹格式（总计21个）进行了比较。我们发现，尽管许多几何特性（距离，RMSD，RDF等）可以从具有较低精度的现有压缩轨迹格式（例如XTC，0.01）复制，但涉及氢的键能对精度损失特别敏感。结果，我们开发了一种基于量化的压缩格式，该格式将原始NetCDF轨迹大小的约66％压缩，其位置精度为5x10-5，具有小于0.1 kcal/mol的能量均方根误差，并且几乎快速读取为原始的未压缩轨迹。 将自动参数下载添加到用于快速PDB至参数生成的软件工具中，用于分子动力学模拟 从实验确定的结构中设置分子动力学模拟通常会因多种因素而复杂化，尤其是当要模拟的结构中包含碳水化合物（例如SARS-COV-2峰值蛋白）时，由于这些碳水化合物具有多种形式，并以多种方式链接。以前，我们开发了一种名为Preparforleap的独立工具，该工具在广泛使用且可自由的软件cpptraj（现已接近4K引用）中实现，该工具促进了使用琥珀色生物分子模拟包的分子动力学模拟的结构的准备。该软件工具是一个独立的程序，不需要Internet访问，几乎没有用户干预，这将其与现有的基于Web的工具区分开来。 通过扩展相似性指标添加聚类与cpptraj 集群分析是数据挖掘技术，可以根据某种相似性的度量来应用于数据点的集合，以创建点组。在MD模拟的背景下，这通常意味着从轨迹中识别重要和独特的构群，这些构构通常包含成千上万到有时数百万个结构。因此，聚类分析是通过降低非常大数据集的维度来指导未来分析的MD模拟分析中非常重要的工具。任何两个点之间成对距离的计算，这可能是瓶颈，缩放尺度很差（O（n2））。 Miranda-Quintana等。最近引入了一种基于确定扩展相似性指数而不是成对距离的新聚类方法，从而将缩放量表降低到O（n）。与他们合作，我们现在将这种新的聚类方法实施到CPPTRAJ中。这将使非常大的数据集对群集分析相对较快。 网格不均匀溶剂化理论计算的并行化 GIST方法的焓部分需要计算网格上的水和水性能。这通常是要点最耗时的部分。现在，我们正在努力通过将其与MPI并行化，从而提高该计算的速度，从而将传入的轨迹框架分配在多个处理器之间。这是特别有吸引力的，因为它在轨迹处理过程中需要很少的单个过程之间的沟通，这意味着计算应很好地扩展到大型处理器计数。除了能量计算外，我们还通过MPI并行化熵计算，这必须在轨迹处理后发生，因为它需要所有轨迹框架中的信息。这种并行的要点方法正在开发中，并将在CPPTRAJ分析软件中免费获得。 cpptraj中时间耗时计算的GPU平行化 我们正在移植几种在多个GPU上同时运行的方法。 收购和硬件升级： NVIDIA A100系统： 在过去的一年中，我们的基础设施进行了战略投资。我们获得了多个系统，专门设计用于增强我们的高性能计算功能。其中包括两个8向NVIDIA A100计算节点和一个4向Nvidia A100计算节点。这些系统配备了多个通过NVLINK连接的多个GPU，NVLink是NVIDIA的尖端GPU至GPU互连技术。 NVIDIA H100系统： 为了证明我们致力于与最新的计算技术保持同步，我们于2023年8月购买了8条NVIDIA H100系统。 这些收购背后的理由是双重的： 可伸缩性： 具有多个GPU的系统的可用性，尤其是当通过NVLink互连时，为我们的团队提供了无与伦比的机会。借助这些系统，我们不仅为当前的工作负载做好准备，而且还支持我们对高级代码库的开发。 专业化： 分子生物学，尤其是通过计算方法接近时，需要高粒度和计算能力。 NVIDIA A100和H100系统是满足我们巨大计算需求的最佳方法。他们承诺增强并行处理能力，更快的内存访问以及提高的数据传输速度，所有这些都对大规模分子模拟和计算至关重要。 总之，过去一年的投资是有条不紊的，针对计算生物学和生物物理学的当前和未来需求。