Collaborative Research: CI-ADDO-EN: Development of Publicly Available, Easily Searchable, Linguistically Analyzed, Video Corpora for Sign Language and Gesture Research

合作研究：CI-ADDO-EN：开发公开可用、易于搜索、语言分析的视频语料库，用于手语和手势研究

• 批准号: 1059235
• 负责人: Vassilis Athitsos
• 金额: $ 6.66万
• 依托单位: University of Texas at Arlington
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2011
• 资助国家: 美国
• 起止时间: 2011-08-01 至 2015-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1059235&HistoricalAwards=false
• 关键词: Collaborative; Research; CI; ADDO; EN

The goal of this project is to create a linguistically annotated, publicly available, and easily searchable corpus of video from American Sign Language (ASL). This will constitute an important piece of infrastructure, enabling new kinds of research in both linguistics and vision-based recognition of ASL. In addition, a key goal is to make this corpus easily accessible to the broader ASL community, including users and learners of ASL. As a result of our long-term efforts, we have an extensive collection of linguistically annotated video data from native signers of ASL. However, the potential value of these corpora has been largely untapped, notwithstanding their extensive and productive use by our team and others. Existing limitations in our hardware and software infrastructure make it cumbersome to search and identify data of interest, and to share data among our institutions and with other researchers. In this project, we propose hardware and software innovations that will constitute a major qualitative upgrade in the organization, searchability, and public availability of the existing (and expanding) corpus. The enhancement and improved Web-accessibility of these corpora will be invaluable for linguistic research, enabling new kinds of discoveries and the testing of hypotheses that would otherwise have be difficult to investigate. On the computer vision side, the proposed new annotations will provide an extensive public dataset for training and benchmarking a variety of computer vision algorithms. This will facilitate research and expedite progress in gesture recognition, hand pose estimation, human tracking, and large vocabulary, and continuous ASL recognition. Furthermore, this dataset will be useful as training and benchmarking data for algorithms in the broader areas of computer vision, machine learning, and similarity-based indexing. The advances in linguistic knowledge about ASL and in computer-based ASL recognition that will be accelerated by the availability of resources of the kind proposed here will contribute to development of technologies for education and universal access. For example, tools for searching collections of ASL video for occurrences of specific signs, or converting ASL signing to English, are still far from attaining the level of functionality and usability to which users are accustomed for spoken/written languages. Our corpora will enable research that aims to bring such vision-based ASL recognition applications closer to reality. Moreover, these resources will afford important opportunities to individuals who would not otherwise be in a position to conduct such research (e.g., for lack of access to native ASL signers or high-quality synchronized video equipment, or lack of resources/expertise to carry out extensive linguistic annotations). Making our corpora available online will also allow the broader community of ASL users to access our data directly. Students of ASL will be able to retrieve video showing examples of a specific sign used in actual sentences, or examples of a grammatical construction. ASL instructors and teachers of the Deaf will also have easy access to video examples of lexical items and grammatical constructions as used by a variety of native signers for use in language instruction and evaluation. Thus, the proposed web interface to our data collection will be a useful educational resource for users, teachers, and learners of ASL.

该项目的目的是创建一个语言注释，公开可用且易于搜索的视频语料库（ASL）。这将构成重要的基础设施，从而在语言学和基于视觉的ASL识别方面具有新的研究。此外，一个关键目标是使更广泛的ASL社区（包括ASL的用户和学习者）轻松访问该语料库。由于我们的长期努力，我们从ASL的本地签名者那里收集了大量语言注释的视频数据。但是，尽管我们的团队和其他人的广泛和富有成效的使用，这些语料库的潜在价值在很大程度上尚未开发。我们的硬件和软件基础架构中的现有局限性使搜索和识别感兴趣的数据并与我们的机构和其他研究人员共享数据很麻烦。在这个项目中，我们提出了硬件和软件创新，这些硬件和软件创新将构成组织的主要定性升级，可搜索性以及现有（和扩展）语料库的公共可用性。这些语料库的增强和提高的Web访问性对于语言研究将是无价的，可以实现新的发现以及对原本难以调查的假设的检验。在计算机视觉方面，拟议的新注释将为培训和基准测试各种计算机视觉算法提供广泛的公共数据集。这将有助于研究和加快手势识别，手姿势估计，人类跟踪以及大量词汇和持续的ASL识别方面的进展。此外，该数据集将有助于在更广泛的计算机视觉，机器学习和基于相似性索引的更广泛领域中训练和基准测试数据。关于ASL和基于计算机的ASL识别的语言知识的进步将通过此处建议的资源的可用性加速，这将有助于发展教育和通用技术的技术。例如，用于搜索ASL视频的收集工具，以了解特定符号的出现或将ASL签名转换为英语，仍然远远无法达到用户习惯于口语/书面语言的功能和可用性水平。我们的语料库将实现旨在将这种基于远景的ASL识别应用程序更接近现实的研究。此外，这些资源将为那些无法进行此类研究的个人提供重要的机会（例如，由于缺乏获得本地ASL签名者或高质量同步的视频设备或缺乏进行广泛语言注释的资源/专业知识）。使我们的语料库在线提供，还可以使更广泛的ASL用户社区直接访问我们的数据。 ASL的学生将能够检索视频，显示实际句子中使用的特定标志的示例，或语法结构的示例。聋人的ASL讲师和教师还可以轻松访问各种本地签名者在语言教学和评估中使用的词汇项目和语法结构的示例。因此，我们数据收集的拟议的Web界面将是ASL的用户，教师和学习者的有用教育资源。