Improving Classifier Confidence using Lossy Label-Invariant Transformations

Providing reliable model uncertainty estimates is imperative to enabling robust decision making by autonomous agents and humans alike. While recently there have been significant advances in confidence calibration for trained models, examples with poor calibration persist in most calibrated models. Consequently, multiple techniques have been proposed that leverage label-invariant transformations of the input (i.e., an input manifold) to improve worst-case confidence calibration. However, manifold-based confidence calibration techniques generally do not scale and/or require expensive retraining when applied to models with large input spaces (e.g., ImageNet). In this paper, we present the recursive lossy label-invariant calibration (ReCal) technique that leverages label-invariant transformations of the input that induce a loss of discriminatory information to recursively group (and calibrate) inputs - without requiring model retraining. We show that ReCal outperforms other calibration methods on multiple datasets, especially, on large-scale datasets such as ImageNet.

提供可靠的模型不确定性估计对于使自主智能体和人类都能做出稳健决策至关重要。虽然最近在对训练模型的置信度校准方面取得了重大进展，但在大多数已校准的模型中，校准不佳的情况仍然存在。因此，人们提出了多种技术，利用输入的标签不变变换（即输入流形）来改善最坏情况下的置信度校准。然而，当应用于具有大输入空间的模型（例如ImageNet）时，基于流形的置信度校准技术通常无法扩展和/或需要昂贵的重新训练。在本文中，我们提出了递归有损标签不变校准（ReCal）技术，该技术利用会导致判别信息丢失的输入标签不变变换来递归地对输入进行分组（和校准）——而不需要重新训练模型。我们表明，ReCal在多个数据集上优于其他校准方法，特别是在像ImageNet这样的大规模数据集上。