FAI: Quantifying and Mitigating Disparities in Language Technologies
FAI:量化和减轻语言技术方面的差异
基本信息
- 批准号:2040926
- 负责人:
- 金额:$ 37.5万
- 依托单位:
- 依托单位国家:美国
- 项目类别:Standard Grant
- 财政年份:2021
- 资助国家:美国
- 起止时间:2021-10-01 至 2023-09-30
- 项目状态:已结题
- 来源:
- 关键词:
项目摘要
Advances in natural language processing (NLP) technology now make it possible to perform many tasks through natural language or over natural language data -- automatic systems can answer questions, perform web search, or command our computers to perform specific tasks. However, ``language'' is not monolithic; people vary in the language they speak, the dialect they use, the relative ease with which they produce language, or the words they choose with which to express themselves. In benchmarking of NLP systems however, this linguistic variety is generally unattested. Most commonly tasks are formulated using canonical American English, designed with little regard for whether systems will work on language of any other variety. In this work we ask a simple question: can we measure the extent to which the diversity of language that we use affects the quality of results that we can expect from language technology systems? This will allow for the development and deployment of fair accuracy measures for a variety of tasks regarding language technology, encouraging advances in the state of the art in these technologies to focus on all, not just a select few.Specifically, this work focuses on four aspects of this overall research question. First, we will develop a general-purpose methodology for quantifying how well particular language technologies work across many varieties of language. Measures over multiple speakers or demographics are combined to benchmarks that can drive progress in development of fair metrics for language systems, tailored to the specific needs of design teams. Second, we will move beyond simple accuracy measures, and directly quantify the effect that the accuracy of systems has on users in terms of relative utility derived from using the system. These measures of utility will be incorporated in our metrics for system success. Third, we focus on the language produced by people from varying demographic groups, predicting system accuracies from demographics. Finally, we will examine novel methods for robust learning of NLP systems across language or dialectal boundaries, and examine the effect that these methods have on increasing accuracy for all users.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
自然语言处理(NLP)技术的进步现在使通过自然语言或自然语言数据执行许多任务成为可能 - 自动系统可以回答问题,执行Web搜索或命令我们的计算机执行特定任务。但是,``语言''并不是整体的。人们在说话的语言,使用的方言,产生语言的相对轻松或选择自己表达自己的单词方面有所不同。然而,在NLP系统的基准测试中,这种语言品种通常没有得到调查。最常见的任务是使用Canonical American English制定的,设计不关心系统是否会在任何其他品种的语言上工作。在这项工作中,我们提出了一个简单的问题:我们可以衡量我们使用的语言多样性的程度会影响我们对语言技术系统所期望的结果的质量吗?这将允许针对各种关于语言技术的任务制定和部署公平的准确度措施,鼓励这些技术的最新进步将重点放在所有人上,而不仅仅是精选的几个。特别是,这项工作集中于这一整体研究问题的四个方面。首先,我们将开发一种通用方法,用于量化特定语言技术在许多语言中的工作方式。多个扬声器或人口统计学的措施与基准相结合,可以推动针对设计团队的特定需求量身定制的语言系统公平指标的进展。其次,我们将超越简单的精度度量,并直接量化系统的准确性对用户的效果,从使用系统中得出的相对效用。这些实用程序的度量将纳入我们的系统成功指标中。第三,我们专注于不同人口群体的人们产生的语言,从而预测了人口统计学的系统精度。最后,我们将研究跨语言或方言边界对NLP系统的强大学习的新颖方法,并研究这些方法对所有用户的准确性提高的效果。该奖项反映了NSF的法定任务,并被认为是值得通过基金会的知识分子优点和更广泛的审查标准来通过评估来通过评估来支持的。
项目成果
期刊论文数量(24)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)
Gendered Mental Health Stigma in Masked Language Models
蒙面语言模型中的性别心理健康耻辱
- DOI:
- 发表时间:2022
- 期刊:
- 影响因子:0
- 作者:Inna Wanyin Lin;Lucille Njoo;Anjalie Field;Ashish Sharma;Katharina Reinecke;Tim Althoff;Yulia Tsvetkov
- 通讯作者:Yulia Tsvetkov
Examining risks of racial biases in NLP tools for child protective services
- DOI:10.1145/3593013.3594094
- 发表时间:2023-05
- 期刊:
- 影响因子:0
- 作者:Anjalie Field;Amanda Coston;Nupoor Gandhi;A. Chouldechova;Emily Putnam-Hornstein;David Steier;Yulia Tsvetkov
- 通讯作者:Anjalie Field;Amanda Coston;Nupoor Gandhi;A. Chouldechova;Emily Putnam-Hornstein;David Steier;Yulia Tsvetkov
SD-QA: Spoken Dialectal Question Answering for the Real World
SD-QA:现实世界的口语方言问答
- DOI:10.18653/v1/2021.findings-emnlp.281
- 发表时间:2021
- 期刊:
- 影响因子:0
- 作者:Faisal, Fahim;Keshava, Sharlina;Alam, Md Mahfuz;Anastasopoulos, Antonios
- 通讯作者:Anastasopoulos, Antonios
Controlled Text Generation as Continuous Optimization with Multiple Constraints
- DOI:
- 发表时间:2021-08
- 期刊:
- 影响因子:0
- 作者:Sachin Kumar;Eric Malmi;Aliaksei Severyn;Yulia Tsvetkov
- 通讯作者:Sachin Kumar;Eric Malmi;Aliaksei Severyn;Yulia Tsvetkov
From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models
- DOI:10.48550/arxiv.2305.08283
- 发表时间:2023-05
- 期刊:
- 影响因子:0
- 作者:Shangbin Feng;Chan Young Park;Yuhan Liu;Yulia Tsvetkov
- 通讯作者:Shangbin Feng;Chan Young Park;Yuhan Liu;Yulia Tsvetkov
共 23 条
- 1
- 2
- 3
- 4
- 5
Graham Neubig其他文献
Non-Native Text-to-Speech Preserving Speaker Individuality Based on Partial Correction of Prosodic and Phonetic Characteristics
基于韵律和语音特征部分校正的非母语文本到语音保留说话人个性
- DOI:10.1587/transinf.2016edp723110.1587/transinf.2016edp7231
- 发表时间:20162016
- 期刊:
- 影响因子:0.7
- 作者:Yuji Oshima;Shinnosuke Takamichi;Tomoki Toda;Graham Neubig;Sakriani Sakti;Satoshi NakamuraYuji Oshima;Shinnosuke Takamichi;Tomoki Toda;Graham Neubig;Sakriani Sakti;Satoshi Nakamura
- 通讯作者:Satoshi NakamuraSatoshi Nakamura
Real-time vibration control of an electrolarynx based on statistical F0 contour prediction
基于统计F0轮廓预测的电喉实时振动控制
- DOI:10.1109/eusipco.2016.776046510.1109/eusipco.2016.7760465
- 发表时间:20162016
- 期刊:
- 影响因子:0
- 作者:Kou Tanaka;Tomoki Toda;Graham Neubig;Satoshi NakamuraKou Tanaka;Tomoki Toda;Graham Neubig;Satoshi Nakamura
- 通讯作者:Satoshi NakamuraSatoshi Nakamura
Attentive Interaction Model: Modeling Changes in View in Argumentation
注意力交互模型:对论证中观点的变化进行建模
- DOI:10.18653/v1/n18-101010.18653/v1/n18-1010
- 发表时间:20182018
- 期刊:
- 影响因子:0
- 作者:Yohan Jo;Shivani Poddar;Byungsoo Jeon;Qinlan Shen;C. Rosé;Graham NeubigYohan Jo;Shivani Poddar;Byungsoo Jeon;Qinlan Shen;C. Rosé;Graham Neubig
- 通讯作者:Graham NeubigGraham Neubig
Enhancing Event-Related Potentials Based on Maximum a Posteriori Estimation with a Spatial Correlation Prior
基于空间相关先验的最大后验估计增强事件相关势
- DOI:10.1587/transinf.2015cbp000810.1587/transinf.2015cbp0008
- 发表时间:20162016
- 期刊:
- 影响因子:0.7
- 作者:Hayato Maki;Tomoki Toda;Sakriani Sakti;Graham Neubig;Satoshi NakamuraHayato Maki;Tomoki Toda;Sakriani Sakti;Graham Neubig;Satoshi Nakamura
- 通讯作者:Satoshi NakamuraSatoshi Nakamura
The NAIST Simultaneous Translation Corpus
NAIST 同声翻译语料库
- DOI:10.1007/978-981-10-6199-8_1110.1007/978-981-10-6199-8_11
- 发表时间:20182018
- 期刊:
- 影响因子:5.8
- 作者:Graham Neubig;Hiroaki Shimizu;S. Sakti;Satoshi Nakamura;T. TodaGraham Neubig;Hiroaki Shimizu;S. Sakti;Satoshi Nakamura;T. Toda
- 通讯作者:T. TodaT. Toda
共 291 条
- 1
- 2
- 3
- 4
- 5
- 6
- 59
Graham Neubig的其他基金
Discovering and Demonstrating Linguistic Features for Language Documentation
发现和展示语言文档的语言特征
- 批准号:17615481761548
- 财政年份:2018
- 资助金额:$ 37.5万$ 37.5万
- 项目类别:Standard GrantStandard Grant
SHF: Small: Open-domain, Data-driven Code Synthesis from Natural Language
SHF:小型:开放域、数据驱动的自然语言代码合成
- 批准号:18152871815287
- 财政年份:2018
- 资助金额:$ 37.5万$ 37.5万
- 项目类别:Standard GrantStandard Grant
RI: EAGER: Collaborative Research: Adaptive Heads-up Displays for Simultaneous Interpretation
RI:EAGER:协作研究:用于同声传译的自适应平视显示器
- 批准号:17486421748642
- 财政年份:2017
- 资助金额:$ 37.5万$ 37.5万
- 项目类别:Standard GrantStandard Grant
相似国自然基金
轻量化多功能因瓦合金多孔材料增材制造与性能表征评价
- 批准号:12372133
- 批准年份:2023
- 资助金额:53 万元
- 项目类别:面上项目
企业层面视角下自由贸易协定条款深度对出口高质量发展的影响:模型拓展与量化分析
- 批准号:72363013
- 批准年份:2023
- 资助金额:27 万元
- 项目类别:地区科学基金项目
明星企业崛起与最优贸易政策制定:理论与量化研究
- 批准号:72303116
- 批准年份:2023
- 资助金额:30 万元
- 项目类别:青年科学基金项目
MRI融合多组学特征量化高级别成人型弥漫性脑胶质瘤免疫微环境并预测术后复发风险的研究
- 批准号:82302160
- 批准年份:2023
- 资助金额:30 万元
- 项目类别:青年科学基金项目
复合场景下基于多模态融合的轻量化任务处理与模型泛化研究
- 批准号:62372111
- 批准年份:2023
- 资助金额:50 万元
- 项目类别:面上项目
相似海外基金
SCH: Quantifying and mitigating demographic biases of machine learning in real world radiology
SCH:量化和减轻现实世界放射学中机器学习的人口统计偏差
- 批准号:1081894110818941
- 财政年份:2023
- 资助金额:$ 37.5万$ 37.5万
- 项目类别:
Quantifying the Impact of the Prescribed Burning on Mitigating Wildland Fire Risk
量化规定燃烧对减轻荒地火灾风险的影响
- 批准号:22308692230869
- 财政年份:2023
- 资助金额:$ 37.5万$ 37.5万
- 项目类别:Standard GrantStandard Grant
QMIA: Quantifying and Mitigating Bias affecting and induced by AI in Medicine
QMIA:量化和减轻人工智能在医学中影响和诱发的偏差
- 批准号:MR/X030075/1MR/X030075/1
- 财政年份:2023
- 资助金额:$ 37.5万$ 37.5万
- 项目类别:Research GrantResearch Grant
Quantifying and Mitigating Vulnerabilities of Residential Structures to Hurricane Risks in Cuba
量化和减轻古巴住宅结构对飓风风险的脆弱性
- 批准号:533758-2018533758-2018
- 财政年份:2020
- 资助金额:$ 37.5万$ 37.5万
- 项目类别:Vanier Canada Graduate Scholarship Tri-Council - Doctoral 3 yearsVanier Canada Graduate Scholarship Tri-Council - Doctoral 3 years
Quantifying and Mitigating Vulnerabilities of Residential Structures to Hurricane Risks in Cuba
量化和减轻古巴住宅结构对飓风风险的脆弱性
- 批准号:533758-2018533758-2018
- 财政年份:2019
- 资助金额:$ 37.5万$ 37.5万
- 项目类别:Vanier Canada Graduate Scholarship Tri-Council - Doctoral 3 yearsVanier Canada Graduate Scholarship Tri-Council - Doctoral 3 years