III: Small: Optimization Techniques for Scalable Semantic Web Data Processing in the Cloud

III：小：云中可扩展语义Web数据处理的优化技术

• 批准号: 1218277
• 负责人: Kemafor Anyanwu-Ogan
• 金额: $ 44.69万
• 依托单位: North Carolina State University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2012
• 资助国家: 美国
• 起止时间: 2012-09-01 至 2017-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1218277&HistoricalAwards=false
• 关键词: III; Small; Optimization; Techniques; Scalable

The use of cloud-based data processing platforms is an increasingly attractive alternative for large-scale data processing. There is active investigation into their use for various types of processing tasks on large-scale unstructured and structured data. However, due to an increased interest in many communities to enable more automatic sharing and exchange of data on the Web using Semantic Web techniques, there is a rapid surge in the availability of very large, real-world, Semantic Web datasets. Such data are semi-structured and have more complex processing requirements than relational data processing due to the fine-grained modeling of data and also the need for inferencing during processing. Consequently, existing optimization techniques for cloud data processing platforms which often adapt relational processing optimization techniques do not address the needs of such workloads. Further, such techniques do not adequately account for the nuances of cloud runtime platforms such as Hadoop e.g., dataflow length as a cost metric, no a-priori existence of indexes and statistics. This project contributes insight into query optimization requirements for Semantic Web data processing on Map Reduce platforms. Its contributions include a novel Nested TripleGroup data model and Algebra (NTGA), algebraic and dynamic cost query optimization techniques; inter and intra-work sharing techniques, data representation formats and system architecture issues of integrating Semantic Web optimization techniques into frameworks such as Apache Pig. The impact of this project will cut across the increasing range of communities that are aggressively adopting Semantic Web tenets such as, scientific, business, government and other general-purpose communities.

用于大规模数据处理的基于云的数据处理平台的使用是越来越有吸引力的替代品。对它们用于大规模非结构化和结构化数据的各种类型的处理任务的使用进行了积极的研究。但是，由于对许多社区的兴趣越来越高，可以使用语义网络技术在网络上进行更多的自动共享和交换数据，因此非常大的，真实的，语义网络数据集的可用性迅速激增。此类数据是半结构化的，并且由于数据的细粒度建模以及在处理过程中的推理需求，因此与关系数据处理相比，关系处理更为复杂。因此，通常适应关系处理优化技术的云数据处理平台的现有优化技术无法满足此类工作负载的需求。此外，此类技术不能充分说明云运行时平台（例如Hadoop）的细微差别，例如DataFly长度作为成本度量，没有索引和统计信息的A-Priori。 该项目有助于深入了解地图上的语义Web数据处理的查询优化要求。它的贡献包括一种新型的嵌套三组数据模型和代数（NTGA），代数和动态成本查询优化技术；工程和内部共享技术，数据表示格式和系统体系结构问题将语义Web优化技术集成到Apache Pig等框架中。该项目的影响将削减越来越多的社区，这些社区正在积极采用语义网络宗旨，例如科学，商业，政府和其他通用社区。