An Empirical Study of the Effect of Source-level Transformations on Compiler Stability

Loop-level compiler optimizations are applied in a complex process with no guarantee that the code produced is optimal. Compilers also struggle to maintain a stable performance on different loops with the same semantics. This paper presents an analysis of the stability of the compilation process and shows potential for state-of-the-art compilers to improve code performance. In the study, loop nests are first extracted from benchmarks; then, sequences of source-level loop transformations are applied to these loop nests to create numerous semantically equivalent mutations; finally, the impact of transformations on code quality in terms of locality, dynamic instruction count, and vectorization is analyzed for different compilers. Our results show that up to 47% of the loops can be improved with at least a 1.15x speedup by this process while the average speedup can reach 1.6x for the improved loops. In addition, we propose a novel stability score that demonstrates the difference in stability from the studied compilers. The study concludes that the effect of source-level transformations varies among compilers, and the evaluated compilers have long ways to go until reaching stable.

循环级编译器优化是在复杂的过程中应用的，不能保证产生的代码是最佳的。编译器还努力通过相同的语义在不同的循环上保持稳定的性能。本文介绍了汇编过程稳定性的分析，并显示了最先进的编译器提高代码性能的潜力。在研究中，首先从基准中提取循环巢。然后，将源级环变换的序列应用于这些环巢，以创建许多语义上等效的突变。最后，对于不同的编译器，分析了转换对当地，动态指导数和矢量化的影响。我们的结果表明，通过此过程至少可以通过至少1.15倍的加速来改进多达47％的循环，而改进的循环的平均速度可以达到1.6倍。此外，我们提出了一个新颖的稳定得分，该得分证明了研究编译器的稳定性差异。该研究得出的结论是，源级变换的影响在编译器之间有所不同，并且评估的编译器还有很长的路要走，直到达到稳定为止。