Collaborative Research: Chameleon: A Large-Scale, Reconfigurable Experimental Environment for Cloud Research

协作研究：Chameleon：用于云研究的大规模、可重构实验环境

• 批准号: 1419141
• 负责人: Katarzyna Keahey
• 金额: $ 248.65万
• 依托单位: University of Chicago
• 依托单位国家: 美国
• 项目类别: Cooperative Agreement
• 财政年份: 2014
• 资助国家: 美国
• 起止时间: 2014-10-01 至 2020-12-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1419141&HistoricalAwards=false
• 关键词: Collaborative; Research; Chameleon; Large; Scale

A persistent problem facing academic cloud research is the lack of infrastructure and data to perform experimental research: large-scale hardware is needed to investigate the scalability of cloud infrastructure and applications, heterogeneous hardware is needed to investigate algorithmic and implementation tradeoffs, fully-configurable software environments are needed to investigate the performance of virtualization techniques and the differences between cloud software stacks, and data about how clouds are used is needed to evaluate virtual machine scheduling and data placement algorithms. The Chameleon project will addresses these needs by providing a large-scale, fully configurable experimental testbed driven by the needs of the cloud research and education communities. The testbed, and the ecosystem associated with it, will enable researchers to explore a range of cloud research challenges, from large scale to small scale, including exploring low-level problems in hardware architecture, systems research, network configuration, and software design, or at higher levels of abstraction looking at cloud scheduling, cloud platforms, and cloud applications. Chameleon will significantly enhance the ability of the computing research community to understand the behavior of Internet scale cloud systems, and to develop new software, ideas and algorithms for the cloud environment. As the tremendous shift to cloud as the primary means of providing computing infrastructure continues, a large-scale testbed tailored to researchers' needs is essential to the continued relevance of a large fraction of computing research. The project is led by the University of Chicago and includes partners from the Texas Advanced Computing Center (TACC), Northwestern University, the Ohio State University, and the University of Texas at San Antonio, comprising a highly qualified and experienced team, with research leaders from the cloud and networking world blended with providers of production quality cyberinfrastructure. The team includes members from the NSF-supported FutureGrid project and from the GENI community, both forerunners of the NSFCloud solicitation under which this project is funded. The Chameleon testbed, will be deployed at the University of Chicago (UC) and the Texas Advanced Computing Center (TACC) and will consist of 650 multi-core cloud nodes, 5PB of total disk space, and leverage 100 Gbps connection between the sites. While a large part of the testbed will consist of homogenous hardware to support large-scale experiments, a portion of it will support heterogeneous units allowing experimentation with high-memory, large-disk, low-power, GPU, and co-processor units. The project will also leverage existing FutureGrid hardware at UC and TACC in its first year to provide a transition period for the existing FutureGrid community of experimental users. To support a broad range of experiments emphasizing a range of requirements ranging from a high degree of control to ease of use the project will support a graduated configuration system allowing full user configurability of the stack, from provisioning of bare metal and network interconnects to delivery of fully functioning cloud environments. In addition, to facilitate experiments, Chameleon will support a set of services designed to meet researchers needs, including support for experimental management, reproducibility, and repositories of trace and workload data of production cloud workloads. To facilitate the latter, the project will form a set of partnerships with commercial as well as academic clouds, such as Rackspace and Open Science Data Cloud (OSDC). It will also partner with other testbeds, notably GENI and INRIA's Grid5000 testbed, and reach out to the user community to shape the policy an direction of the testbed. The Chameleon project will bring a new dimension and scale of resources to the CS community who wish to educate their students about design, implementation, operation and applications of cloud computing, a critical skillset for future computing professionals. It will enhance the understanding and application of experimental methodology in computer science and generate new educational materials and resources, with the participation of, and for, Minority Serving Institution (MSI) students.

学术云研究面临的一个长期问题是缺乏进行实验研究的基础设施和数据：需要大规模硬件来研究云基础设施和应用程序的可扩展性，需要异构硬件来研究算法和实现权衡，需要完全可配置的软件需要环境来研究虚拟化技术的性能以及云软件堆栈之间的差异，并且需要有关如何使用云的数据来评估虚拟机调度和数据放置算法。 Chameleon 项目将通过提供由云研究和教育社区的需求驱动的大规模、完全可配置的实验测试平台来满足这些需求。该测试台以及与之相关的生态系统将使研究人员能够探索一系列从大规模到小规模的云研究挑战，包括探索硬件架构、系统研究、网络配置和软件设计中的低级问题，或者在更高的抽象级别上查看云调度、云平台和云应用程序。 Chameleon 将显着增强计算研究社区了解互联网规模云系统行为的能力，并为云环境开发新的软件、想法和算法。随着向云计算作为提供计算基础设施的主要手段的巨大转变仍在继续，根据研究人员的需求量身定制的大规模测试床对于大部分计算研究的持续相关性至关重要。该项目由芝加哥大学牵头，包括来自德克萨斯高级计算中心（TACC）、西北大学、俄亥俄州立大学和德克萨斯大学圣安东尼奥分校的合作伙伴，组成了一支高素质、经验丰富的团队，研究带头人来自云和网络世界与生产质量网络基础设施提供商的融合。 该团队包括来自 NSF 支持的 FutureGrid 项目和 GENI 社区的成员，他们都是 NSFCloud 征集的先驱，该项目是在该项目下资助的。 Chameleon 测试床将部署在芝加哥大学 (UC) 和德克萨斯高级计算中心 (TACC)，由 650 个多核云节点、5PB 的总磁盘空间组成，并利用站点之间的 100 Gbps 连接。虽然测试台的很大一部分将由同质硬件组成以支持大规模实验，但其中一部分将支持异构单元，允许使用高内存、大磁盘、低功耗、GPU 和协处理器单元进行实验。该项目还将在第一年利用 UC 和 TACC 现有的 FutureGrid 硬件，为现有的 FutureGrid 实验用户社区提供一个过渡期。为了支持广泛的实验，强调从高度控制到易用性的一系列要求，该项目将支持分级配置系统，允许用户对堆栈进行全面的配置，从提供裸机和网络互连到交付功能齐全的云环境。此外，为了促进实验，Chameleon 将支持一组旨在满足研究人员需求的服务，包括对实验管理、可重复性以及生产云工作负载的跟踪和工作负载数据存储库的支持。为了促进后者，该项目将与商业和学术云建立一系列合作伙伴关系，例如 Rackspace 和开放科学数据云 (OSDC)。它还将与其他测试平台合作，特别是 GENI 和 INRIA 的 Grid5000 测试平台，并接触用户社区来制定测试平台的政策和方向。 Chameleon 项目将为希望教育学生有关云计算的设计、实施、操作和应用的计算机科学社区带来新的维度和规模的资源，这是未来计算专业人员的关键技能。它将加强对计算机科学实验方法的理解和应用，并在少数族裔服务机构 (MSI) 学生的参与下并为少数族裔服务机构 (MSI) 学生提供新的教育材料和资源。