Collaborative Research: Adaptive Data Parallel Storage

协作研究：自适应数据并行存储

• 批准号: 0090221
• 负责人: Evgenia Smirni
• 金额: $ 18.54万
• 依托单位: College of William and Mary
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2001
• 资助国家: 美国
• 起止时间: 2001-10-01 至 2005-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0090221&HistoricalAwards=false
• 关键词: Collaborative; Research; Adaptive; Data; Parallel

The I/O demands of scientific applications are increasing exponentially as researchers strive for greater accuracy in models and simulations of physical systems, and as more information about the world is obtained. To deal with this aspect of computation implies that one cannot ignore the I/O gap and only concentrate on processor-centric issues. As a concrete example, the size of Genbank, a nucleotide database, has been doubling every 14 months and is currently 33 GB. In the commercial arena attention has turned to information and its location also. More and more network services, fueled by the demands of information-intensive Web applications, are shifting the focus in computer systems to I/O.With the growing importance of data comes the need to address its management. The commodity cluster of the near future, which consists of a combination of network-attached storage, intelligent data appliances and workstations, and high-performance networks, is a highly parallel, heterogeneous computer. However, the file abstractions in use today cannot yet fully exploit either the resulting computational power or data locality.This award examines how to improve I/O performance on such a platform for a class of important applications. UC Santa Cruz has a leading computational biology group with a 100+ node Linux cluster; the parallel sequencing codes that they use are an obvious target. Other groups have also expressed interest in this cluster; and its use will be an excellent source of applications. There are three main research objectives. First, develop an I/O programming model that unites computation and storage. This will allow the programmer to express "storage operations" that can be readily parallelized. Second, build an adaptive infrastructure and develop analytic models to determine the optimal execution path, which may be parallel or sequential. Third, develop offline models to answer questions about how to allocate resources within this heterogeneous environment.The goal in creating a new I/O interface is to associate computation and data so that code can be easily executed at the source of the data. For example, suppose we want to search for a word in a dictionary. The standard procedure would be to open the dictionary file, read it in chunks, search for instances of the word in each chunk, and close the file. This operation is sequential and processor-centric; the emphasis is on moving the data to the processor rather than a computation to the data. Instead, we propose to associate a "search for word" command with the dictionary in the form of a new I/O interface. Now the "search for word" becomes a high level abstraction for the same code described above, a single remote procedure call, or a parallel procedure (depending on the location of the dictionary).

随着研究人员在物理系统的模型和模拟中努力提高准确性，并且获得有关世界的更多信息，科学应用的I/O需求呈指数增长。 处理计算的这一方面意味着人们不能忽略I/O间隙，而只专注于以处理器为中心的问题。作为一个具体的例子，核苷酸数据库的GenBank的大小每14个月增加一倍，目前为33 GB。 在商业舞台上，注意力也转向了信息及其位置。信息密集型Web应用程序的需求推动了越来越多的网络服务，将计算机系统中的重点转移到I/O中。随着数据的重要性越来越重要，需要解决其管理。不久的将来的商品集群由网络连接存储，智能数据设备和工作站以及高性能网络组合组成，是一款高度平行的，异构的计算机。但是，当今使用中的文件摘要还不能完全利用由此产生的计算能力或数据局部性。该奖项研究如何在此类平台上提高I/O性能的一类重要应用程序。 UC Santa Cruz拥有一个领先的计算生物学组，其中有100+节点Linux群集；它们使用的并行测序代码是明显的目标。其他小组也对此集群表示兴趣。它的使用将是应用程序的绝佳来源。有三个主要的研究目标。首先，开发一个I/O编程模型，该模型将计算和存储统一。这将允许程序员表达可以容易平行的“存储操作”。其次，构建一个自适应基础架构并开发分析模型以确定最佳的执行路径，这可能是并行或顺序的。第三，开发离线模型，以回答有关如何在此异质环境中分配资源的问题。创建新的I/O接口的目标是关联计算和数据，以便可以在数据源上轻松执行代码。例如，假设我们想在字典中搜索一个单词。标准过程是打开字典文件，在块中读取它，在每个块中搜索单词的实例，然后关闭文件。此操作是顺序的，以处理器为中心；重点是将数据移至处理器而不是将数据转移到数据上。相反，我们建议以新的I/O接口的形式将“搜索Word”命令与字典相关联。现在，“搜索单词”成为上述相同代码的高级抽象，单个远程过程调用或并行过程（取决于字典的位置）。