A Study on Checkpoint Compression for Adjoint Computation

— When we want to understand the sensitivity of a simulation model with respect to an input value or to optimize an objective function, the gradient usually provides a good hint. The adjoint state method is a widely used numerical method to compute the gradient of a function. It decomposes functions into a sequence of basic operations. It performs a forward sweep to evaluate the function, followed by a backward sweep to calculate the gradient using the chain rule iteratively. One limitation of the adjoint state method is that all intermediate values from the forward sweep are needed by the backward sweep. Usually, we keep only a portion of those values, called checkpoints, in the memory because of limited space. The remaining values are either stored on the hard disk or recomputed from the nearest checkpoint whenever needed. In this work, we seek to compress the intermediate values in order to better utilize limited space in the memory and to speed the I/O when checkpointing to the hard disk.

- 当我们想要了解仿真模型对输入值的敏感性或优化目标函数时，梯度通常提供了很好的提示。伴随状态法是计算函数梯度的一种常用数值方法。它将函数分解为一系列基本操作。它执行向前扫描以计算函数，然后执行向后扫描以使用链式规则迭代地计算梯度。伴随状态方法的一个局限性是，前向扫描的所有中间值都需要后向扫描。通常，由于空间有限，我们只在内存中保留这些值的一部分，称为检查点。其余的值要么存储在硬盘上，要么在需要时从最近的检查点重新计算。在这项工作中，我们寻求压缩中间值，以更好地利用有限的空间在内存中，并加快I/O时，检查点到硬盘。