RI: Small: Early Elementary Reading Verification in Challenging Acoustic Environments

RI：小：具有挑战性的声学环境中的早期小学阅读验证

• 批准号: 2008043
• 负责人: Eric Fosler-Lussier
• 金额: $ 45万
• 依托单位: Ohio State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-08-01 至 2024-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2008043&HistoricalAwards=false
• 关键词: RI; Small; Early; Elementary; Reading

Young students who have difficulty in reading fluently can gain confidence by repeatedly reading story passages. Computer reading assessment can help improve student’s fluency while freeing up teacher to help students more efficiently. In previous projects, computer-based technologies have been shown to improve the reading ability of low performing readers in urban schools. Reading verification is a critical task, determining whether a student has correctly read the words on the page, or has mispronounced words, is stuck, or has jumped ahead or back. However, current systems that use automatic speech recognition (ASR) tend towards accepting incorrect pronunciations, instead of figuring out mistakes, and operate poorly when there is noise. This project addresses these issues by studying how to build a reliable computer system that can tell, in real-time, whether students are reading correctly within a noisy classroom, and make it available on many different types of computers. To allow other researchers to compare their studies, the project team will create and make available a dataset of reading examples recorded in noise through the Language Research Lab at the Center of Science and Industry (COSI) museum. By interacting with lab personnel, visitors will be educated on how speech recognition systems work as well as on how language science research is done. Thus, if this project is successful, significantly better computer-based reading assessment technologies will be developed, which will help many children read better, and the public will become better informed about reading, speech technologies and science in general. The approach investigated in this project abandons the need for full ASR technology, instead performing Reading Verification (RV) directly on the speech signal while jointly performing speech enhancement to focus on the child's speech. The elimination of ASR decoding technology permits creation of smaller verification models that can be deployed on devices or web browsers. Building on pilot work in mispronunciation detection and integration of phonetic information into speech enhancement, a new attention-based tracking approach is examined that directly models phenomena like partial word disfluencies, explores the difference between enhancement for child and adult speech, and investigates a novel joint model of detection and enhancement integrating longitudinal adaptation techniques. The research activities are supported by a new collection of speech data, consisting of stories read in a noisy science museum environment. The recorded dataset will facilitate comparative work in reading assessment and will be released as part of the Ohio Child Speech Corpus.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在流利阅读方面存在困难的年轻学生可以通过重复阅读故事段落来获得信心，从而帮助提高学生的流利程度，同时让老师能够更有效地帮助学生。在之前的项目中，基于计算机的技术已被证明可以提高阅读能力。阅读验证是城市学校中表现不佳的读者能力的一项关键任务，确定学生是否正确阅读了页面上的单词，是否发音错误，是否卡住，或者向前或向后跳跃。自动语音识别（ASR）倾向于接受不正确的发音，而不是找出错误，并且在有噪音时运行不佳。该项目通过研究如何构建一个可靠的计算机系统来解决这些问题，该系统可以实时判断学生是否正在阅读。为了让其他研究人员能够比较他们的研究，项目团队将通过中心的语言研究实验室创建并提供在噪音中记录的阅读示例数据集。科学与工业 (COSI) 博物馆。通过与实验室人员互动，参观者将了解语音识别系统如何工作以及语言科学研究如何进行。因此，如果该项目成功，将开发出更好的基于计算机的阅读评估技术，这将帮助许多人。孩子们的阅读能力更好，公众也将更好地了解阅读、语音技术和科学。该项目研究的方法放弃了对完整 ASR 技术的需求，而是在联合执行时直接对语音信号进行阅读验证 (RV)。言语增强要注重孩子言语的消除。 ASR 解码技术允许创建可部署在设备或网络浏览器上的较小验证模型，基于发音错误检测和将语音信息集成到语音增强中的试点工作，研究了一种新的基于注意力的跟踪方法，该方法可以直接对部分现象进行建模。单词不流利性，探讨了儿童和成人语音增强之间的差异，并研究了一种集成纵向适应技术的新型检测和增强联合模型。这些研究活动得到了一组新的语音数据的支持，这些数据包括在嘈杂的科学中阅读的故事。博物馆环境。记录的数据集将促进阅读评估的比较工作，并将作为俄亥俄州儿童语音语料库的一部分发布。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力优点和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Fine-Grained Textual Knowledge Transfer to Improve RNN Transducers for Speech Recognition and Understanding

细粒度文本知识转移可改进 RNN 转换器以实现语音识别和理解

• DOI: 10.1109/icassp49357.2023.10094997
• 发表时间: 2023-06
• 期刊: Speech and Signal Processing
• 作者: Sunder, Vishal;Thomas, Samuel;Kuo, Hong;Kingsbury, Brian;Fosler
• 通讯作者: Fosler

End-to-End Word-Level Disfluency Detection and Classification in Children’s Reading Assessment

儿童阅读评估中的端到端单词级不流畅检测和分类

• DOI: 10.1109/icassp49357.2023.10095555
• 发表时间: 2023-06-04
• 期刊: ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
• 作者: Lavanya Venkatasubramaniam;Vishal Sunder;E. Fosler
• 通讯作者: E. Fosler