III: Small: Optimization Techniques for Scalable Semantic Web Data Processing in the Cloud

III：小：云中可扩展语义Web数据处理的优化技术

• 批准号: 1218277
• 负责人: Kemafor Anyanwu-Ogan
• 金额: $ 44.69万
• 依托单位: North Carolina State University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2012
• 资助国家: 美国
• 起止时间: 2012-09-01 至 2017-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1218277&HistoricalAwards=false
• 关键词: III; Small; Optimization; Techniques; Scalable

The use of cloud-based data processing platforms is an increasingly attractive alternative for large-scale data processing. There is active investigation into their use for various types of processing tasks on large-scale unstructured and structured data. However, due to an increased interest in many communities to enable more automatic sharing and exchange of data on the Web using Semantic Web techniques, there is a rapid surge in the availability of very large, real-world, Semantic Web datasets. Such data are semi-structured and have more complex processing requirements than relational data processing due to the fine-grained modeling of data and also the need for inferencing during processing. Consequently, existing optimization techniques for cloud data processing platforms which often adapt relational processing optimization techniques do not address the needs of such workloads. Further, such techniques do not adequately account for the nuances of cloud runtime platforms such as Hadoop e.g., dataflow length as a cost metric, no a-priori existence of indexes and statistics. This project contributes insight into query optimization requirements for Semantic Web data processing on Map Reduce platforms. Its contributions include a novel Nested TripleGroup data model and Algebra (NTGA), algebraic and dynamic cost query optimization techniques; inter and intra-work sharing techniques, data representation formats and system architecture issues of integrating Semantic Web optimization techniques into frameworks such as Apache Pig. The impact of this project will cut across the increasing range of communities that are aggressively adopting Semantic Web tenets such as, scientific, business, government and other general-purpose communities.

使用基于云的数据处理平台是大规模数据处理越来越有吸引力的替代方案。人们正在积极研究它们在大规模非结构化和结构化数据上的各种类型的处理任务中的用途。然而，由于许多社区对使用语义 Web 技术在 Web 上实现更自动的数据共享和交换越来越感兴趣，因此非常大的真实世界语义 Web 数据集的可用性迅速激增。此类数据是半结构化的，由于数据的细粒度建模以及处理过程中需要进行推理，因此比关系数据处理具有更复杂的处理要求。因此，通常采用关系处理优化技术的云数据处理平台的现有优化技术不能满足此类工作负载的需求。此外，此类技术没有充分考虑 Hadoop 等云运行时平台的细微差别，例如，数据流长度作为成本指标，没有索引和统计数据的先验存在。 该项目有助于深入了解 MapReduce 平台上语义 Web 数据处理的查询优化要求。其贡献包括新颖的嵌套 TripleGroup 数据模型和代数 (NTGA)、代数和动态成本查询优化技术；将语义 Web 优化技术集成到 Apache Pig 等框架中的工作间和工作内共享技术、数据表示格式和系统架构问题。该项目的影响将涉及越来越多积极采用语义网原则的社区，例如科学、商业、政府和其他通用社区。