CSR-PDOS: ISG: Collaborative Research: Building distributed, wide-area applications using WheelFS

CSR-PDOS：ISG：协作研究：使用 WheelFS 构建分布式广域应用程序

基本信息

批准号：
0720029
负责人：
Marinus Kaashoek
金额：
$ 35万
依托单位：
Massachusetts Institute of Technology
依托单位国家：
美国
项目类别：
Continuing Grant
财政年份：
2007
资助国家：
美国
起止时间：
2007-09-01 至 2011-08-31
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=0720029&HistoricalAwards=false
关键词：
CSR PDOS ISG Collaborative Research

项目摘要

Large-scale applications distributed across thousands of machines over the wide area network are difficult to build and maintain. To enable sharing of data across machines, many applications use specialized storage and data transfer tools such as a DHT, scp or GridFTP. Although file systems are successful in becoming a common building block for cluster applications, it remains unclear if file systems could provide similar benefits to wide area distributed applications. This research describes a novel wide-area file system, WheelFS, that allows distributed application developers to use a generic file system interface to store and share application data easily among wide-area machines. Two new approaches make WheelFS attractive for use by distributed applications. First, WheelFS provides semantic cues for application developers to express desired tradeoffs among failure resilience, data consistency and file system performance at the granularity of individual files and directories.Second, WheelFS optimizes wide area data transfer by writing application data to local disks and reading a cached copy from a nearby machine whenever possible.This project demonstrates the uselessness of WheelFS via the experience of building a number of distributed applications, such as a cooperative web cache, a data-intensive Grid application, a distributed digital library and aPlanetLab measurement utility.

通过广域网分布在数千台机器上的大型应用程序很难构建和维护。为了实现跨机器共享数据，许多应用程序使用专门的存储和数据传输工具，例如 DHT、scp 或 GridFTP。尽管文件系统成功地成为集群应用程序的通用构建块，但仍不清楚文件系统是否可以为广域分布式应用程序提供类似的好处。这项研究描述了一种新颖的广域文件系统，WheelFS，它允许分布式应用程序开发人员使用通用文件系统接口在广域机器之间轻松存储和共享应用程序数据。两种新方法使 WheelFS 对分布式应用程序具有吸引力。首先，WheelFS 为应用程序开发人员提供语义提示，以在单个文件和目录的粒度上表达故障恢复、数据一致性和文件系统性能之间的所需权衡。其次，WheelFS 通过将应用程序数据写入本地磁盘并读取数据来优化广域数据传输。尽可能从附近的机器缓存副本。该项目通过构建许多分布式应用程序（例如协作式 Web 缓存、数据密集型网格应用程序、分布式数字图书馆和 aPlanetLab 测量）的经验，展示了 WheelFS 的无用性公用事业。