Efficient and Fair Language Modelling for Natural Language Processing, investigating lightweight language modelling approaches and aiming at fairness

自然语言处理的高效公平语言建模，研究轻量级语言建模方法并以公平为目标

• 批准号: 2894795
• 金额: --
• 依托单位: University of Sheffield
• 依托单位国家: 英国
• 项目类别: Studentship
• 财政年份: 2023
• 资助国家: 英国
• 起止时间: 2023 至 无数据
• 项目状态: 未结题
• 来源: https://gtr.ukri.org/projects?ref=studentship-2894795
• 关键词: Efficient; Fair; Language; Modelling; Natural

In natural language processing (NLP), pre-trained language models, such as BERT, and GPT-3 provide the de facto units for processing languages, but have expensive training costs and hardware requirements like Graphics Processing Units (GPUs), and there are questions about their interpretability and fairness. This research will investigate lightweight language modelling approaches and methods to account for fairness in the learning process. The goal is to enable the training of language model in an efficient manner, contributing to environmentally-friendly NLP research and widening access to research communities with limited computational resources. These goals are particularly timely in light of recent legislation and regulations regarding the use of artificial intelligence (AI)-based systems around the world.

在自然语言处理（NLP）中，预先训练的语言模型（例如BERT和GPT-3）提供了用于处理语言的事实上的单元，但具有昂贵的培训成本和硬件要求（例如图形处理单元（GPU））有关其解释性和公平性的问题。这项研究将研究轻巧的语言建模方法和方法，以说明学习过程中的公平性。目的是以有效的方式实现语言模型的培训，从而有助于环保的NLP研究，并以有限的计算资源来扩大对研究社区的访问。鉴于最近有关使用人工智能（AI）基于世界各地的系统的法规和法规，这些目标尤其及时。