CSR: Small: Algorithms and Abstractions for Efficient Virtual-Memory Streaming and Big-Data Computing

CSR：小：高效虚拟内存流和大数据计算的算法和抽象

• 批准号: 1717982
• 负责人: Dmitri Loguinov
• 金额: $ 47.1万
• 依托单位: Texas A&M Engineering Experiment Station
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-09-01 至 2021-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1717982&HistoricalAwards=false
• 关键词: CSR; Small; Algorithms; Abstractions; Efficient; CSR; Small; Algorithms; Abstractions; Efficient

As the field stands today, operating systems provide a poor interface for data-intensive computing, requiring programmers to engage in tedious, non-reconfigurable, and error-prone code development. These software-engineering practices often lead to easily exploitable vulnerabilities and devastating security breaches. This project builds various algorithms and implementations for a virtual-stream interface that enables our society to develop big-data software that is more easily managed, simpler to understand, inherently faster, and less buggy. Operating systems have used virtual memory and paging for decades; however, user-level applications are still required to process input/output in blocks of fixed size. This project takes a different approach by creating a zero-copy streaming abstraction that offers sequential access to bulk data with unprecedented simplicity, flexibility, and speed. The outcomes of this research not only improve the internal functionality of operating systems and hardware, but also permit reuse of existing libraries in external-memory operation, lead to significantly faster in-place sorting and inter-thread communication, and pave the way to more scalable database computing. The project delivers novel system-level concepts and prototypes that simplify algorithm design, enable faster processing of large-scale data streams, reduce software cost, and help produce better technology for the 21st century. The project also engages students at Texas A&M University in research-intensive education in cross-disciplinary fields, broadens integration of fundamental research into classroom teaching, mentors students, and permits related research in the industry and institutions around the world through publicly shared outcomes of our work.Project data will be maintained online for as long as it is feasible. The shared products include publications, data, software, and various research artifacts. The project URL is http://irl.cs.tamu.edu/projects/streams/

如今，该领域如今，操作系统为数据密集型计算提供了差的界面，要求程序员参与乏味，不可恢复和易于错误的代码开发。这些软件工程实践通常会导致易于利用的漏洞和毁灭性的安全漏洞。该项目为虚拟流界面构建了各种算法和实现，该界面使我们的社会能够开发更容易管理的大数据软件，更容易管理，更简单地理解，固有的速度更快且较小的货物。几十年来，操作系统使用了虚拟内存和分页； 但是，仍然需要用户级应用程序来处理固定尺寸块中的输入/输出。该项目通过创建零拷贝流抽象来采用不同的方法，该抽象以前所未有的简单性，灵活性和速度提供顺序访问批量数据。这项研究的结果不仅改善了操作系统和硬件的内部功能，而且还允许在外部记忆操作中重复使用现有库，从而导致在本地分类和线程间通信中显着更快，并为更可扩展的数据库计算铺平道路。 该项目提供了新颖的系统级概念和原型，这些概念和原型简化了算法设计，可以更快地处理大型数据流，降低软件成本并帮助为21世纪提供更好的技术。该项目还吸引了德克萨斯A＆M大学的学生在跨学科领域的研究密集型教育，扩大对课堂教学的基本研究的融合，指导学生，并通过我们的工作公开共享的工作来维持在线上的公开共享。共享产品包括出版物，数据，软件和各种研究工具。项目URL是http://irl.cs.tamu.edu/projects/streams/