STCI: Development of Stork Data Scheduler for Mitigating the Data Bottleneck in Petascale Distributed Computing Systems

STCI：开发 Stork 数据调度程序，以缓解千万亿级分布式计算系统中的数据瓶颈

• 批准号: 0926701
• 负责人: Joel Tohline
• 金额: $ 49.55万
• 依托单位: Louisiana State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2009
• 资助国家: 美国
• 起止时间: 2009-09-01 至 2013-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0926701&HistoricalAwards=false
• 关键词: STCI; Development; Stork; Data; Scheduler

This proposal will be awarded using funds made available by the American Recovery and Reinvestment Act of 2009 (Public Law 111-5), and meets the requirements established in Section 2 of the White House Memorandum entitled, Ensuring Responsible Spending of Recovery Act Funds, dated March 20, 2009.The STCI Development of Stork Data Scheduler for Mitigating the Data Bottleneck in Petascale Distributed Computing Systems project will enhance the Stork data scheduler to mitigate the end-to-end data handling bottleneck in petascale distributed computing systems and make it available for a wide range of user community as production quality software. New functionalities of Stork will include: data aggregation and caching; early error detection, classification, and recovery; integration with workflow planning and management; optimal protocol tuning; and data transfer performance prediction services.Intellectual Merit:The Stork data scheduler will make a distinctive contribution to petascale distributed computing in the areas of planning, scheduling, monitoring and management of data placement tasks and application-level end-to-end optimization of networked I/O for petascale distributed applications. Unlike existing approaches, it will treat data resources and the tasks related to data access and movement as first class entities just like computational resources and compute tasks, and not simply the side effect of computation. Broader Impact:This project will impact not just traditionally compute intensive disciplines from science and engineering, but also new emerging computational areas in the arts, humanities, business and education. The PI will be collaborating with other leading institutions in the area of distributed data management such as LBNL, ISI/USC, UNC, UCSD, and University Chicago/Argonne to integrate Stork with their data management solutions and disseminate it to their user communities. The comprehensive education component of the project will include science projects and summer training camps on data-intensive computing with K-12 students (where 99% are minority students), undergraduate and graduate student training, international student/intern exchange program, and minority workshops.

该提案将使用2009年《美国恢复和再投资法》（公法111-5）提供的资金颁发，并满足白宫备忘录第2节中确定的要求，该要求，确保了2009年3月20日的恢复恢复法案的负责支出，日期为2009年3月20日。减轻Petascale分布式计算系统中的端到端数据处理瓶颈，并使其可用于广泛的用户社区作为生产质量软件。 Stork的新功能将包括：数据聚合和缓存；早期错误检测，分类和恢复；与工作流计划和管理集成；最佳协议调整； Intlectual功绩：Stork数据调度程序将对Petascale分布式计算做出独特的贡献。与现有方法不同，它将将数据资源和与数据访问和移动相关的任务视为一流的实体，就像计算资源和计算任务一样，而不仅仅是计算的副作用。更广泛的影响：该项目不仅会影响传统上从科学和工程学中计算密集的学科，而且会影响艺术，人文，商业和教育的新新兴计算领域。 PI将与分布式数据管理领域的其他领先机构合作，例如LBNL，ISI/USC，UNC，UCSD和芝加哥/Argonne，以将Stork与他们的数据管理解决方案集成并将其传播给其用户社区。该项目的综合教育部分将包括与K-12学生（99％是少数群体学生），本科和研究生培训，国际学生/实习生交流计划以及少数派研讨会有关的科学项目和夏季培训训练营（99％是少数群体学生）。