RAPID: Advanced Topic Modeling Methods to Analyze Text Responses in COVID-19 Survey Data

RAPID：用于分析 COVID-19 调查数据中文本响应的高级主题建模方法

• 批准号: 2031736
• 负责人: Philip Resnik
• 金额: $ 17.68万
• 依托单位: University of Maryland, College Park
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-05-15 至 2023-04-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2031736&HistoricalAwards=false
• 关键词: RAPID; Advanced; Topic; Modeling; Methods

As the COVID-19 pandemic continues, public and private organizations are deploying surveys to inform responses and policy choices. Survey designs using multiple choice responses are by far the most common -- "open ended" questions, where survey participants provide a longer-form written response, are used far less. This is true despite the fact that when you allow people to provide unconstrained spoken or text responses, it is possible to obtain richer, fine-grained information clarifying the other responses, as well as useful “bottom up” information that the survey designers did not know to ask for. A key problem is that analyzing the unstructured language in open-ended responses is a labor-intensive process, creating obstacles to using them especially when speedy analysis is needed and resources are limited. Computational methods can help, but they often fail to provide coherent, interpretable categories, or they can fail to do a good job connecting the text in the survey with the closed-end responses. This project will develop new computational methods for fast and effective analysis of survey data that includes text responses, and it will apply these methods to support organizations doing high-impact survey work related to COVID-19 response. This will improve these organizations’ ability to understand and mitigate the impact of the COVID-19 pandemic.This project’s technical approach builds on recent techniques bringing together deep learning and Bayesian topic models. Several key technical innovations will be introduced that are specifically geared toward improving the quality of information available in surveys that include both closed- and open-ended responses. A common element in these approaches is the extension of methods commonly used in supervised learning settings, such as task-based fine-tuning of embeddings and knowledge distillation, to unsupervised topic modeling, with a specific focus on producing diverse, human-interpretable topic categories that are well aligned with discrete attributes such as demographic characteristics, closed-end responses, and experimental condition. Project activities include assisting in the analysis of organizations' survey data, conducting independent surveys aligned with their needs to obtain additional relevant data, and the public release of a clean, easy to use computational toolkit facilitating more widespread adoption of these new methods.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

随着Covid-19的大流行的继续，公共和私人组织正在部署调查以告知回应和政策选择。使用多项选择响应的调查设计是迄今为止最常见的 - “开放式”问题，调查参与者提供了更长形式的书面响应，使用的使用要少得多。这是真正的目的地，当您允许人们提供不受限制的口语或文本响应时，可以获得更丰富，细粒度的信息来阐明其他响应，并获得调查设计师不知道要要求的有用的“自下而上”信息。一个关键问题是，在开放式响应中分析的非结构化语言是一个实验室密集型过程，造成了使用它们的障碍，尤其是在需要快速分析并且资源受到限制时。计算方法可以提供帮助，但是它们通常无法提供连贯的，可解释的类别，或者他们无法在将调查中的文本与闭环响应中连接起来做得很好。该项目将开发新的计算方法，以快速有效地分析包括文本响应的调查数据，并将这些方法应用这些方法来支持从事与COVID-19响应有关的高影响力调查工作的组织。该项目的技术方法基于最近的技术，将深度学习和贝叶斯主题模型融合在一起。将引入几项关键的技术创新，专门针对提高调查中可用信息的质量，包括封闭式和开放式响应。这些方法中的一个共同元素是扩展在监督学习设置中常用的方法，例如基于任务的嵌入和知识蒸馏的微调来进行无监督的主题建模，其特定的重点是生产潜水员，人性化的主题类别，这些类别与诸如否定性的属性良好，例如诸如诸如粉料特征，封闭式的条件和实验性，实验性，实验性。项目活动包括协助分析组织的调查数据，进行独立的调查，符合其需要获得其他相关数据的需求，以及公众发布清洁，易于使用的计算工具包，支持对这些新方法的更广泛采用。该奖项反映了NSF的法定任务，并通过评估了基金会的智力效果，以诚实地对其进行评估。

项目成果

Is Automated Topic Model Evaluation Broken?: The Incoherence of Coherence

• 发表时间: 2021-07
• 作者: Alexander Miserlis Hoyle;Pranav Goel;Denis Peskov;Andrew Hian-Cheong;Jordan L. Boyd-Graber;P. Resnik

Improving Neural Topic Models Using Knowledge Distillation

• DOI: 10.18653/v1/2020.emnlp-main.137
• 发表时间: 2020-10
• 作者: Alexander Miserlis Hoyle;Pranav Goel;P. Resnik