Evaluating and automating the annotation of a learner corpus

The paper describes a corpus of texts produced by non-native speakers of Czech. We discuss its annotation scheme, consisting of three interlinked tiers, designed to handle a wide range of error types present in the input. Each tier corrects different types of errors; links between the tiers allow capturing errors in word order and complex discontinuous expressions. Errors are not only corrected, but also classified. The annotation scheme is tested on a data set including approx. 175,000 words with fair inter-annotator agreement results. We also explore the possibility of applying automated linguistic annotation tools (taggers, spell checkers and grammar checkers) to the learner text to support or even substitute manual annotation.

本文描述了一个由非捷克母语者所创作的文本语料库。我们讨论了其标注方案，该方案由三个相互关联的层级组成，旨在处理输入文本中存在的多种错误类型。每个层级纠正不同类型的错误；层级之间的关联能够捕捉语序错误以及复杂的非连续表达。错误不仅得到纠正，还被分类。该标注方案在一个约含17.5万字的数据集上进行了测试，标注者间一致性结果良好。我们还探讨了将自动语言标注工具（词性标注器、拼写检查器和语法检查器）应用于学习者文本以支持甚至替代人工标注的可能性。