SEI(GEO): Scientific Data Operators Optimized for Efficient Distributed Interactive and Batch Analysis of Tera-Scale Geophysical Data

SEI(GEO)：针对兆级地球物理数据的高效分布式交互式和批量分析而优化的科学数据运算符

• 批准号: 0431203
• 负责人: Charles Zender
• 金额: $ 59.44万
• 依托单位: University of California-Irvine
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2004
• 资助国家: 美国
• 起止时间: 2004-09-01 至 2008-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0431203&HistoricalAwards=false
• 关键词: SEI; GEO; Scientific; Data; Operators

Many geophysical simulations and nearly all climate models archive their results in hundreds to thousands of files spanning gigabytes to terabytes of storage. Before the geophysicist can interpret these data, it must be reduced to a manageable size (e.g., by averaging it) and/or transported to a specified location (e.g., the desktop computer). Currently researchers often copy raw data over the Internet from multiple locations to commence their data analysis. This project is developing the Scientific Data Operators (SDO): efficient software that lets researchers perform typical data reduction and analysis in parallel, remotely, without wasting time and network bandwidth.The SDO software combines distributed and shared memory programming, client-server architecture, and Open Source development techniques. As proof-of-concept, a distributed analysis of multiple NCAR CCSM IPCC climate assessment simulations (each is about a terabyte) within and across national boundaries will be performed. This project also provides support for a graduate student to carry out dissertation research in distributed climate analysis. Outside climate modeling and analysis, SDO will have three main impacts: (1) increase the value of large geophysical datasets by decreasing the time to analyze, discover, and publish new results; (2) reveal any critical bandwidth, I/O, and client/server bottlenecks in processing distributed geophysical data; and (3) improve analysis of growing bioinformatics data sets, especially gene expression data, in ways similar to the geophysics domain. SDO is free software based on the internationally successful netCDF Operator (NCO) software. The project results including this software will be accessible via the project web site.

许多地球物理模拟和几乎所有气候模型都将其结果归纳为数百到数千个千兆字节到储存的文件。在地球物理学家可以解释这些数据之前，必须将其降低为可管理的大小（例如，通过平均）和/或运输到指定位置（例如桌面计算机）。目前，研究人员经常通过Internet从多个位置复制原始数据，以开始其数据分析。该项目正在开发科学数据运营商（SDO）：有效的软件，使研究人员可以同时进行典型的数据减少和分析，而无需浪费时间和网络带宽。SDO软件结合了分布式和共享的内存编程，客户端服务器架构，端子服务器架构，和开源开发技术。作为概念验证，将进行多个NCAR CCSM IPCC气候评估模拟（每个）内部和跨国家边界内的分布式分析。该项目还为研究生在分布式气候分析中进行论文研究提供了支持。外部气候建模和分析，SDO将产生三个主要影响：（1）通过减少分析，发现和发布新结果的时间来增加大型地球物理数据集的价值； （2）在处理分布式地球物理数据时揭示任何关键的带宽，I/O和客户端/服务器瓶颈； （3）改善对生物信息学数据集（尤其是基因表达数据）的分析，其方式与地球物理领域相似。 SDO是基于国际成功的NETCDF操作员（NCO）软件的免费软件。 包括该软件在内的项目结果将通过项目网站访问。