CAREER: Speedy and Reliable Approximate Queries in Hybrid Transactional/Analytical Systems

职业：混合事务/分析系统中快速可靠的近似查询

• 批准号: 2339596
• 负责人: Zhuoyue Zhao
• 金额: $ 60万
• 依托单位: SUNY at Buffalo
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2024
• 资助国家: 美国
• 起止时间: 2024-05-15 至 2029-04-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2339596&HistoricalAwards=false
• 关键词: CAREER; Speedy; Reliable; Approximate; Queries

Real-time data analytics allow one to extract timely insights from today’s large and rapidly growing databases, which can provide important economic and social values. Examples include fraud detection using online financial transaction data, optimizing marketing strategies based on analysis of real time data, etc. A new type of database system called Hybrid Transactional/Analytical Processing (HTAP) is built to perform these analytical queries over online transactional databases with low response time, but they require increasing computation resources and may still have prolonged query response time as the data continue to grow rapidly. Approximate Query Processing (AQP) techniques can significantly reduce query response time by performing random sampling in the query processing pipelines, but they are only designed for static databases that cannot be updated online. This project seeks to support scalable real-time data analytics on large and rapidly growing databases, by enabling speedy and reliable AQP capabilities in HTAP systems. The project will result in an open-source system that supports approximate real-time data analytics, and thus can potentially enable the aforementioned real-time data analytics applications. Furthermore, this project will also support the development of new educational materials on modern data management systems, include HTAP and AQP systems, as well as research training of undergraduate and graduate students, to improve the readiness of the STEM workforce. In addition, it will also support development of educational materials in data management for K-12 outreach programs and improve the public awareness of database technologies.Existing HTAP systems perform exact query processing, which incurs at least linear computation cost to input size, and are no longer a viable solution as the rapid growth of data has outpaced limited increase in processor speed and storage bandwidth. Approximate Query Processing (AQP) is a fast alternative that may achieve sublinear time cost if the application can tolerate approximation, but the existing techniques suffer from several drawbacks including high data scan cost, inability to perform correct and efficient transactional updates, as well as inaccurate estimation and unreliable error diagnosis results. This project aims to resolve these drawbacks through a co-design of AQP and HTAP system components including data storage and indexing layer, transaction concurrency control protocols and approximate query processing algorithms. Specifically, this project will result in three main scientific contributions: (1) It will develop a thread-safe, high-performance, and succinct sampling index design for HTAP storage. It will provide the necessary thread-safe atomic update and fast sampling capabilities for enabling speedy and reliable AQP in HTAP systems. (2) It will design new protocols to enforce snapshot isolation and serializability for database transactions with mixed updates and approximate queries. (3) It will also investigate a new sampling strategy leveraging the fast-sampling capabilities to minimize approximate query latency given a user-specified confidence bound target, and a background diagnosis service for reliably diagnosing estimation failures where the true answer does not fall into the estimated confidence interval with the user-specified confidence.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

实时数据分析使人们可以从当今大型且快速增长的数据库中提取及时的见解，这可以提供重要的经济和社会价值。示例包括使用在线财务交易数据进行欺诈检测，基于对实时数据的分析等优化营销策略。一种称为混合交易/分析处理（HTAP）的新型数据库系统旨在对在线交易数据库进行以低响应时间的方式执行这些分析查询，但是它们需要越来越多的计算资源，但它们仍然需要增加响应量的响应时间，以延长QUERY QUERY aff agry af agraply the agrip and aprip and agriping aprip and aprip and agrip and the数据。近似查询处理（AQP）技术可以通过在查询处理管道中执行随机采样来大大减少查询响应时间，但它们仅为无法在线更新的静态数据库而设计。该项目旨在通过在HTAP系统中启用快速且可靠的AQP功能来支持大型且快速增长的数据库的可扩展实时数据分析。该项目将产生一个支持近似实时数据分析的开源系统，因此可以潜在地实现近似的实时数据分析应用程序。此外，该项目还将支持开发有关现代数据管理系统的新教育材料，包括HTAP和AQP系统，以及对本科生和研究生的研究培训，以提高STEM劳动力的准备。此外，它还将支持开发K-12外展计划数据管理中的教育材料，并提高公众对数据库技术的认识。存在HTAP系统执行精确的查询处理，这至少可以将线性计算成本提高到输入尺寸，并且不再是可行的解决方案，因为数据的快速增长超过了流程速度和存储量的速度限制的速度和存储量的速度增加。近似查询处理（AQP）是一种快速的替代方案，如果应用程序可以忍受近似，则可能会达到额定时间的成本，但是现有技术遭受了多种缺点，包括高数据扫描成本，无法执行正确，有效的交易更新，以及不准确的估计以及不可靠的错误诊断结果。该项目旨在通过AQP和HTAP系统组件的共同设计，包括数据存储和索引层，交易并发控制协议以及近似查询处理算法。特别是，该项目将产生三个主要的科学贡献：（1）它将开发出螺纹效果，高度储存和独立索引samsexs设计，该项目将产生三个主要的科学贡献。它将提供必要的线程安全原子更新和快速采样功能，以在HTAP系统中启用快速且可靠的AQP。 （2）它将设计新协议，以使用混合更新和近似查询来实施数据库交易的快照隔离和序列化。 （3）它还将调查一种新的抽样策略，以利用快速采样功能，以最小化近似查询延迟，并在用户指定的置信度约束目标上以及可靠的诊断估计失败的背景诊断服务，而真实答案并未通过用户指定的置信度进行估计的置信度。优点和更广泛的影响审查标准。