Identifying Semantically Difficult Samples to Improve Text Classification

In this paper, we investigate the effect of addressing difficult samples from a given text dataset on the downstream text classification task. We define difficult samples as being non-obvious cases for text classification by analysing them in the semantic embedding space; specifically - (i) semantically similar samples that belong to different classes and (ii) semantically dissimilar samples that belong to the same class. We propose a penalty function to measure the overall difficulty score of every sample in the dataset. We conduct exhaustive experiments on 13 standard datasets to show a consistent improvement of up to 9% and discuss qualitative results to show effectiveness of our approach in identifying difficult samples for a text classification model.

在本文中，我们研究了处理给定文本数据集中困难样本对下游文本分类任务的影响。我们通过在语义嵌入空间中对样本进行分析，将困难样本定义为文本分类中不明显的情况；具体来说——（i）属于不同类别的语义相似样本以及（ii）属于同一类别的语义不相似样本。我们提出了一个惩罚函数来衡量数据集中每个样本的总体难度得分。我们在13个标准数据集上进行了详尽的实验，结果显示性能持续提高了高达9%，并且讨论了定性结果，以证明我们的方法在为文本分类模型识别困难样本方面的有效性。