CSR: Small: Elastic Soft State Cache as an OS Service

CSR：小型：弹性软状态缓存作为操作系统服务

• 批准号: 2330831
• 负责人: Harry Xu
• 金额: $ 60万
• 依托单位: University of California-Los Angeles
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2024
• 资助国家: 美国
• 起止时间: 2024-01-01 至 2026-12-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2330831&HistoricalAwards=false
• 关键词: CSR; Small; Elastic; Soft; State

To efficiently serve user requests, cloud applications cache soft-state data in memory. Data is considered “soft state” when it is helpful only for efficiency and losing it would not lead to functional issues. Soft state is ubiquitously used as it enables applications to trade off memory for better performance, and performance gains often grow with the amount of memory provisioned. Today’s systems provision memory statically to applications to prevent their cached data from growing unbounded and draining the server’s memory. It is extremely hard, however, for static memory provisioning to be able to address each application’s soft-state needs while simultaneously maximizing the use of global memory. Dynamic provisioning is challenging as current systems are unable to safely and efficiently add/remove memory between applications. This project builds a new virtual memory abstraction for soft-state caching (referred to as “soft memory’”), as well as a soft memory management system called Midas, enabling application developers to easily register their soft-state data with the Midas API and leave cache management and dynamic scaling to the OS kernel. Under Midas, soft-state caching is no longer application-specific; instead, it is provided by the OS kernel as a service. Modern society is relying increasingly on cloud platforms where memory is heavily underutilized. On the application side, most cloud applications need to cache intermediate results or soft-state data; more memory can significantly boost their throughput and reduce their latency. However, existing cloud systems provision memory statically, leading to memory waste. Simultaneously, the caching needs of many memory-hungry applications are not being satisfied. This project develops a new soft-state caching system, providing caching to all applications as an operating system service. Such a service can significantly improve memory utilization for the cloud, leading to monetary cost savings. In the meantime, it improves the performance of individual applications running on the cloud by letting them opportunistically use more memory, resulting in improved user experiences. The project also makes efforts in education and diversity by incorporating research into courses and recruiting and training junior researchers from underrepresented groups.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

为了有效地服务用户请求，云应用程序将软状态数据缓存在内存中，当数据仅对效率有帮助并且丢失它不会导致功能问题时，它会被广泛使用，因为它使应用程序能够执行以下操作。牺牲内存以获得更好的性能，并且性能增益通常会随着所配置的内存量的增加而增长，以防止其缓存数据无限制地增长并耗尽服务器的内存，但这对于静态内存来说非常困难。供应能够满足每个应用程序的软状态需求，同时最大限度地利用全局内存是具有挑战性的，因为当前系统无法安全有效地在应用程序之间添加/删除内存。该项目为软状态构建了新的虚拟内存抽象。 -状态缓存（称为“软内存”），以及称为 Midas 的软内存管理系统，使应用程序开发人员能够轻松地使用 Midas API 注册其软状态数据，并将缓存管理和动态扩展留给操作系统内核下。 Midas，软状态缓存不再是特定于应用程序的；相反，它是由操作系统内核作为服务提供的。在应用程序方面，大多数云应用程序都需要充分利用内存。缓存中间结果或软状态数据；更多的内存可以显着提高其吞吐量并减少延迟。但是，现有的云系统静态提供内存，同时导致许多内存密集型应用程序的缓存需求得不到满足。 .这个项目开发了一种新的软状态缓存系统，将缓存作为操作系统服务提供给所有应用程序，这样的服务可以显着提高云的内存利用率，从而节省资金成本，同时提高单个应用程序运行的性能。该项目还通过将研究纳入课程以及招募和培训来自代表性不足群体的初级研究人员，在教育和多样性方面做出了努力。该奖项反映了 NSF 的法定使命，并得到了认可。被认为值得通过使用基金会的智力优势和更广泛的影响审查标准进行评估来获得支持。