CSR: Small: Lightning in Clouds: Detection and Characterization of Very Short Bottlenecks

CSR：小：云中闪电：极短瓶颈的检测和表征

• 批准号: 1421561
• 负责人: Calton Pu
• 金额: $ 45万
• 依托单位: Georgia Tech Research Corporation
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2014
• 资助国家: 美国
• 起止时间: 2014-10-01 至 2017-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1421561&HistoricalAwards=false
• 关键词: CSR; Small; Lightning; Clouds; Detection

A plausible explanation for the persistent low utilization of data centers (around 18% by Gartner reports) is the managerial need to maintain quality of service against the well-known Latency Long Tail problem, where some apparently random requests that normally return within milliseconds would suddenly take multiple seconds. The latency long tail problem arises at moderate utilization levels (e.g., 50%) with all resources far from saturation. Despite the efforts to remedy the latency long tail problem in various ways, its causes have remained elusive: In most cases, the very requests that took several seconds actually return within milliseconds when executed by themselves. Studying and solving the latency long tail problem will contribute to better utilization while maintaining quality of service, leading to lower costs for cloud users, higher return on investment for cloud providers, and lower power consumption for the environment. The main goal of this project is the investigation of the class of very short bottlenecks, in which the CPU becomes saturated only for a small fraction of a second, as a significant cause of latency long tail problems. Despite their short lifespan, very short bottlenecks can lead to significant response time increases (several seconds) by propagating queuing effects up and down the request chain in an n-tier application system because of strong dependencies among the tiers during request processing. This project runs large scale experiments in clouds and simulators to generate extensive fine-grain monitoring data in the investigation of very short bottlenecks, which are virtually invisible under typical performance monitoring tools with sampling periods of seconds or minutes. To match the time scale of very short bottlenecks, special instrumentation software tools are being refined to sample intra-server resource utilization at millisecond resolution and timestamp inter-server messages at microsecond resolution. Preliminary studies of n-tier application benchmarks with naturally bursty workloads have found very short bottlenecks that cause latency long tail in several system layers: systems software (JVM garbage collection), processor architecture (dynamic voltage and frequency scaling), and consolidation of applications in virtualized cloud environments. They show the potential for many other sources of very short bottlenecks, e.g., kernel daemon processes that use 100% of CPU for several milliseconds. Through careful distributed event analysis of the experimental data, new kinds of very short bottlenecks can be discovered, verified, reproduced, and studied in detail. Concrete solutions for specific very short bottlenecks have been developed, e.g., an improved Java garbage collector. However, other very short bottlenecks have no specific bug-fixes, e.g., those created by consolidated workload overlapping bursts of statistical nature. As an alternative to bug-fixes, more general solutions that disrupt queuing propagation are being explored. As a concrete example, instead of using a classic request/response approach, where waiting threads participate in the queuing propagation, asynchronous requests with notification of responses to reduce overall queuing is being investigated as a potential solution to eliminate or reduce the impact of several kinds of very short bottlenecks.

数据中心利用率持续低下（Gartner 报告约为 18%）的一个合理解释是，管理需要针对众所周知的延迟长尾问题来维持服务质量，即一些通常在几毫秒内返回的明显随机请求会突然出现。需要几秒钟。延迟长尾问题出现在中等利用率水平（例如 50%）且所有资源远未饱和的情况下。尽管人们努力以各种方式解决延迟长尾问题，但其原因仍然难以捉摸：在大多数情况下，花费几秒钟的请求在自行执行时实际上会在几毫秒内返回。研究和解决延迟长尾问题将有助于提高利用率，同时保持服务质量，从而降低云用户的成本，为云提供商提供更高的投资回报，并降低环境的功耗。该项目的主要目标是调查非常短的瓶颈类别，其中 CPU 仅在一小部分时间内达到饱和，这是延迟长尾问题的一个重要原因。尽管它们的生命周期很短，但非常短的瓶颈可能会导致响应时间显着增加（几秒钟），因为在请求处理过程中各层之间存在很强的依赖性，因此在 n 层应用程序系统中的请求链中上下传播排队效应，从而导致响应时间显着增加（几秒）。该项目在云和模拟器中运行大规模实验，以在调查非常短的瓶颈时生成广泛的细粒度监控数据，这些瓶颈在采样周期为几秒或几分钟的典型性能监控工具下几乎是不可见的。为了匹配非常短的瓶颈的时间尺度，正在改进特殊的仪器软件工具，以毫秒分辨率对服务器内资源利用率进行采样，并以微秒分辨率对服务器间消息进行时间戳处理。对具有自然突发工作负载的 n 层应用程序基准的初步研究发现，非常短的瓶颈会在多个系统层中导致延迟长尾：系统软件（JVM 垃圾收集）、处理器架构（动态电压和频率缩放）以及应用程序整合虚拟化云环境。它们显示了许多其他非常短的瓶颈来源的潜力，例如，内核守护进程在几毫秒内使用 100% 的 CPU。通过对实验数据进行仔细的分布式事件分析，可以发现、验证、重现和详细研究新型的非常短的瓶颈。针对特定的非常短的瓶颈的具体解决方案已经开发出来，例如改进的 Java 垃圾收集器。然而，其他非常短的瓶颈没有特定的错误修复，例如，由统计性质的合并工作负载重叠突发造成的瓶颈。作为错误修复的替代方案，正在探索破坏排队传播的更通用的解决方案。举一个具体的例子，我们正在研究使用带有响应通知的异步请求来减少总体排队的潜在解决方案，而不是使用等待线程参与排队传播的经典请求/响应方法，作为消除或减少多种影响的潜在解决方案。非常短的瓶颈。