RI: Medium: Collaborative Research: Models of Handshape Articulatory Phonology for Recognition and Analysis of American Sign Language

RI：媒介：协作研究：用于识别和分析美国手语的手形发音音系模型

• 批准号: 1409886
• 负责人: Diane Brentari
• 金额: $ 27.55万
• 依托单位: University of Chicago
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2014
• 资助国家: 美国
• 起止时间: 2014-06-01 至 2019-05-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1409886&HistoricalAwards=false
• 关键词: RI; Medium; Collaborative; Research; Models

Sign languages are the primary means of communication for millions of Deaf people in the world, including about 350,000-500,000 American Sign Language (ASL) users in the US. While the hearing population has benefited from advances in speech technologies such as speech recognition and spoken web search, much less progress has been made for sign language interfaces. Advances depend on improved technology for analyzing sign language from video. In addition, the linguistics of sign language is less well-understood than that of spoken language. This project addresses both of these needs, with an interdisciplinary approach that will contribute to research in linguistics, language processing, computer vision, and machine learning. Applications of the work include better access to ASL social media video archives, interactive recognition and search applications for Deaf individuals, and ASL-English interpretation assistance.This project focuses on handshape in ASL, in particular on one constrained but very practical component: fingerspelling, or the spelling out of a word as a sequence of handshapes and trajectories between them. Fingerspelling comprises up to 35% of ASL, depending on the context, and includes 72% of ASL handshapes, making it an excellent testing ground. The project addresses gaps in existing work by focusing on handshape in various conditions, including fast, highly coarticulated signing. The main project activities include development of (1) robust automatic detection and recognition of fingerspelled words using new handshape models, including segmental and "multi-segmental" graphical models of ASL phonological features; (2) techniques for generalizing across signers, styles, and recording conditions; (3) improved phonetics and phonology of handshape, in particular contributing to an articulatory phonology of sign; and (4) publicly released multi-speaker, multi-style fingerspelling data and associated semi-automatic annotation.

手语是世界上数百万聋人的主要交流方式，其中包括美国约 350,000-500,000 名美国手语 (ASL) 用户。 虽然听力人群受益于语音识别和语音网络搜索等语音技术的进步，但手语界面的进展却要少得多。 进步取决于分析视频中手语的技术的改进。 此外，手语的语言学不如口语的语言学那么容易理解。 该项目通过跨学科方法满足了这两个需求，将有助于语言学、语言处理、计算机视觉和机器学习的研究。 这项工作的应用包括更好地访问 ASL 社交媒体视频档案、针对聋人的交互式识别和搜索应用程序以及 ASL-英语翻译帮助。该项目重点关注 ASL 中的手形，特别是一个受限制但非常实用的组成部分：手指拼写、或者将单词拼写为一系列手形和手形之间的轨迹。 根据上下文，手指拼写占 ASL 的比例高达 35%，并且包括 72% 的 ASL 手形，使其成为绝佳的测试场。 该项目通过关注各种条件下的手形（包括快速、高度协同的手语）来解决现有工作中的空白。 主要项目活动包括开发（1）使用新的手形模型对手指拼写单词进行鲁棒的自动检测和识别，包括 ASL 语音特征的分段和“多分段”图形模型； (2) 跨手语者、风格和录音条件进行概括的技术； (3) 改进了手形的语音学和音韵学，特别是有助于符号的发音音韵学； (4)公开发布的多说话人、多风格的手指拼写数据及相关的半自动标注。