SBIR Phase I: Real-Time Artificial Intelligence (AI) Bidirectional American Sign Language (ASL) Communication System

SBIR第一阶段：实时人工智能（AI）双向美国手语（ASL）通信系统

• 批准号: 2213235
• 负责人: Nicholas Wilkins
• 金额: $ 25.6万
• 依托单位: SIGN-SPEAK Inc
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-02-15 至 2024-05-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2213235&HistoricalAwards=false
• 关键词: SBIR; Phase; Real; Time; Artificial

The broader impact of this Small Business Innovation Research (SBIR) Phase I project is to improve the communication between Deaf and Hard of Hearing (D/HH) individuals and the hearing community through automated sign language recognition. In the United States alone there are over 48 million D/HH individuals, who in total possess $87 billion in purchasing power. It appears businesses are not adequately serving this community, as is evidenced by the plethora of Americans with Disabilities Act (ADA) lawsuits against numerous companies. The proposed technology will provide plug-and-play software for organizations to improve their interactions with D/HH individuals. Businesses and governments will be able to interact with their D/HH employees, customers, or constituents when interpreters are unavailable. This technology can be integrated into a variety of platforms, from retail point-of-sale equipment to chatbots and video/teleconferencing systems.This Small Business Innovation Research (SBIR) Phase 1 project aims to develop technology to perform unconstrained sign language recognition and natural sign language production. Specifically, current methods to train language translation models are ill-equipped to handle the sign language domain due to the lack of training data within this domain. Additionally, all currently established methods (apart from motion capture, which is unscalable) for producing American Sign Language (ASL) result in stilted, unnatural signing from an avatar. This project will develop solutions to these issues within the domain of ASL via semi-supervised expert-augmented models and data augmentation techniques. Technical hurdles include the lack of models to handle high-dimensional low-resource language domains, and lack of sufficiently large datasets. Technical milestones include creating semi-supervised datasets, engineering data augmentation techniques, generating a natural signing avatar, and performing extensive usability testing. This project aims to produce a method for automatically interpreting between a low-resource sign language and English to improve accessibility and increase equity for the Deaf and Hard of Hearing communities.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

这项小型企业创新研究（SBIR）I阶段项目的更广泛影响是通过自动化的手语识别来改善聋哑和听力（d/hh）个人与听力社区之间的沟通。仅在美国，就有超过4800万d/hh的人，他们拥有870亿美元的购买力。看来企业没有充分为这个社区服务，这是针对众多公司的《美国残疾人法》（ADA）诉讼的多项诉讼所证明的。拟议的技术将为组织提供插件软件，以改善其与D/HH个人的互动。当口译员不可用时，企业和政府将能够与D/HH员工，客户或成员互动。该技术可以集成到各种平台中，从零售销售点设备到聊天机器人以及视频/电信系统系统。这项小型企业创新研究（SBIR）第1阶段项目旨在开发技术以执行不受限制的手语语言知识和自然手势语言生产。具体而言，由于该域内缺乏训练数据，目前的训练语言翻译模型无法处理手语领域的方法不足。此外，目前所有建立的方法（除运动捕获（不可计算）以外，无法产生美国手语（ASL），从而从阿凡达尔（Avatar）产生了不自然的签名。该项目将通过半监督专家授权的模型和数据增强技术在ASL领域内开发解决方案。技术障碍包括缺乏处理高维低资源语言领域的模型，以及缺乏足够大的数据集。技术里程碑包括创建半监督数据集，工程数据增强技术，生成自然签名化身以及进行广泛的可用性测试。该项目旨在生产一种方法，以自动解释低资源手语和英语之间，以提高聋人和听力社区的公平性，并提高听力社区的公平性。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子优点和更广泛的审查标准通过评估来获得支持的。