EAGER:High Performance Algorithms for Interactive Data Science at Scale

EAGER：大规模交互式数据科学的高性能算法

• 批准号: 2109988
• 负责人: David Bader
• 金额: $ 18.74万
• 依托单位: New Jersey Institute of Technology
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-03-01 至 2025-06-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2109988&HistoricalAwards=false
• 关键词: EAGER; Performance; Algorithms; Interactive; Data

A real-world challenge in data science is to develop interactive methods for quickly analyzing new and novel data sets that are potentially of massive scale. This award will design and implement fundamental algorithms for high performance computing solutions that enable the interactive large-scale data analysis of massive data sets. Based on the widely-used data types and structures of strings, sets, matrices and graphs, this methodology will produce efficient and scalable software for three classes of fundamental algorithms that will drastically improve the performance on a wide range of real-world queries or directly realize frequent queries. These innovations will allow the broad community to move massive-scale data exploration from time-consuming batch processing to interactive analyses that give a data analyst the ability to comprehensively, deeply and efficiently explore the insights and science in real world data sets. By enabling the increasing number of developers to easily manipulate large data sets, this will greatly enlarge the data science community and find much broader use in new communities. Materials from this project will be included in graduate and undergraduate course curriculum. Especially, women, high school students and other underrepresented groups in STEM areas will be encouraged to participate in this research activity. This project focuses on these three important data structures for data analytics: 1) suffix array construction, 2) 'treap' construction and 3) distributed memory join algorithms, useful for analyzing large scale strings, implementing random search in large string data sets, and generating new relations, respectively. These fundamental algorithms serve as the cornerstone to support interactive data science at scale. Based on the theoretical achievements and systematic algorithm design, a novel symbiotic optimization methodology that can combine the theoretical analysis, data structure features, and typical data distribution features together as a whole will be developed to significantly improve the practical performance of the proposed algorithms. To evaluate and show the effectiveness of the proposed algorithms, these algorithms will be implemented in and contribute to an open source NumPy-like software framework that aims to provide productive data discovery tools on massive, dozens-of-terabytes data sets by bringing together the productivity of Python with world-class high performance computing.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

数据科学中的一个现实挑战是开发交互式方法，以快速分析潜在规模的新型和新颖的数据集。该奖项将针对高性能计算解决方案设计和实施基本算法，以实现大规模数据集的交互式大规模数据分析。基于广泛使用的数据类型和字符串，集合，矩阵和图形的结构，该方法将为三类基本算法生成有效且可扩展的软件，这些软件将大大改善广泛的现实查询中的性能或直接实现频繁查询。这些创新将使广泛的社区从耗时的批处理处理中将大规模的数据探索转移到交互式分析中，这些分析使数据分析师能够全面，深刻，深刻，高效地探索现实世界数据集中的见解和科学。通过使越来越多的开发人员轻松地操纵大型数据集，这将大大扩大数据科学界，并在新社区中找到更广泛的用途。 该项目的材料将包括在研究生和本科课程课程中。尤其是，将鼓励妇女，高中生和其他代表性不足的群体参加这项研究活动。该项目着重于数据分析的这三个重要数据结构：1）后缀阵列构造，2）“ Treap”构建和3）分布式内存加入算法，可用于分析大规模字符串，在大型字符串数据集中实施随机搜索，并分别生成新的关系。这些基本算法是支持互动数据科学的基石。基于理论成就和系统的算法设计，一种新型的共生优化方法可以将理论分析，数据结构特征和典型的数据分布特征整体结合在一起，以显着提高所提出算法的实践性能。为了评估和显示拟议算法的有效性，这些算法将实施并为开源的类似Numpy的软件框架而实施，并旨在为大规模的数据发现工具提供富有生产力的数据发现工具，数十个数据集，数十个数据集，通过将python的生产力与世界级的授权授予。利用基金会的知识分子和更广泛的影响审查标准。

项目成果

Triangle Counting Through Cover-Edges

通过盖边缘进行三角形计数

• 发表时间: 2023
• 期刊: The 27th Annual IEEE High Performance Extreme Computing Conference (HPEC
• 作者: Bader, David;Li, Fuhuan;Ganeshan, Anya;Gundogdu, Ahmet;Lew, Jason;Alvarado Rodriguez, Oliver;Du, Zhihui
• 通讯作者: Du, Zhihui

Triangle Centrality in Arkouda

Arkouda 的三角形中心性

• 发表时间: 2022
• 期刊: The 26th Annual IEEE High Performance Extreme Computing Conference (HPEC
• 作者: Joseph Patchett;Zhihui Du;Fuhuan Li;David A. Bader
• 通讯作者: David A. Bader

Anti-Section Transitive Closure

• DOI: 10.1109/hipc53243.2021.00033
• 发表时间: 2021-12
• 期刊: 2021 IEEE 28th International Conference on High Performance Computing, Data, and Analytics (HiPC)
• 作者: Oded Green;Zhihui Du;Sanyamee Patel;Zehui Xie;Hang Liu;David A. Bader

Parallel Longest Common SubSequence Analysis In Chapel

• DOI: 10.1109/hpec58863.2023.10363472
• 发表时间: 2023-09
• 期刊: 2023 IEEE High Performance Extreme Computing Conference (HPEC)
• 作者: Soroush Vahidi;B. Schieber;Zhihui Du;David A. Bader

Fast Triangle Counting

快速三角形计数

• 发表时间: 2023
• 期刊: The 27th Annual IEEE High Performance Extreme Computing Conference (HPEC
• 作者: Bader, David