Collaborative Research: Multimethod Investigation of Articulatory and Perceptual Constraints on Natural Language Evolution

合作研究：自然语言进化的发音和感知约束的多方法研究

基本信息

批准号：
1749376
负责人：
Matthew Dye
金额：
$ 34.4万
依托单位：
Rochester Institute of Tech
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2018
资助国家：
美国
起止时间：
2018-05-15 至 2023-10-31
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1749376&HistoricalAwards=false
关键词：
Collaborative Research Multimethod Investigation Articulatory

项目摘要

Languages change over time, such that the way we speak English now is very different than the speech patterns of elder generations and our distant ancestors. This project will exploit the visual nature of sign languages--where the body parts producing language are highly visible--to determine whether languages change so that they are easier to produce or so that they are easier to understand. In doing so, the project will address fundamental theoretical questions about language change that cannot be addressed by analyzing historical samples of spoken languages. To this end, the researchers will develop computational tools that allow 3D human body poses to be automatically extracted from 2D video. Such tools will be useful for the development of automated sign language recognition, promoting accessibility for deaf and hard-of-hearing people, and for developing automated systems for recognizing and classifying human gestures. The research will involve deaf and hard-of-hearing students, helping to increase diversity in the nation's scientific workforce.It is well documented that sign languages change over time, and it is a commonly held belief that those changes have resulted from successive generations making language easier to perceive. However, most of this evidence has been anecdotal and descriptive and has not quantified changes in the ease of perception and production of ASL over time. The research team will take advantage of the fully visible articulators of sign languages to develop novel pose estimation algorithms that are able to automatically extract information contained in 2D video to create accurate 3D models of articulator movement during language production. The recent birth and rapid evolution of Nicaraguan Sign Language (NSL) has allowed researchers to study language change, from the beginning, on a compressed time-scale. By leveraging an existing NSL database--comprised of 2D videos from four generations of Nicaraguan signers--and utilizing these novel pose estimation algorithms, the researchers will be able to empirically assess the extent to which linguistic changes are driven by perceptual constraints imposed by the human visual system and/or articulatory constraints imposed by the musculoskeletal system. The researchers will also query lexical databases of American Sign Language to test predictions about the perceptual form of modern day ASL, and conduct behavioral studies with deaf and hearing users of ASL to test hypotheses regarding the allocation of visual attention as a result of both deafness and acquisition of a sign language. In doing so, the research will provide valuable information about how the human brain changes the tools we use (in this case, language) and the way that those tools in turn shape the function of the human brain. This will provide a more complex understanding of language change that illuminates the complex interaction between languages and the human beings that use them.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

随着时间的流逝，语言会随着时间的流逝而改变，使我们现在说英语的方式与老年一代和我们遥远的祖先的语音模式大不相同。该项目将利用标志语言的视觉性质（在这里产生语言的人体部分是高度可见的）来确定语言是否在变化，以便它们更易于产生或使其更易于理解。通过这样做，该项目将解决有关语言变化的基本理论问题，而这无法通过分析口语的历史样本来解决。为此，研究人员将开发允许从2D视频中自动提取3D人体的计算工具。此类工具将有助于开发自动手语识别，促进聋人和听力障碍者的可及性，以及开发自动化系统以识别和分类人类手势。这项研究将涉及聋哑学生和听力障碍的学生，有助于增加美国科学劳动力的多样性。有充分的文献证明，符号语言会随着时间的流逝而改变，这通常是一种信念，即这些变化是由于后代而导致的，使语言易于感知。但是，大多数这些证据都是轶事和描述性的，并且没有量化随着时间的推移的感知和产生的易感性变化。研究团队将利用标志语言的完全可见的旋转器来开发新颖的姿势估计算法，这些算法能够自动提取2D视频中包含的信息，以创建语言生产过程中准确的3D旋转器运动模型。尼加拉瓜手语（NSL）的最近出生和快速发展使研究人员从一开始就可以在压缩的时间表上研究语言变化。通过利用来自四代尼加拉瓜签名者的2D视频的现有NSL数据库，并利用这些新颖的姿势估计算法，研究人员将能够评估人类视觉系统和/或ARNACERS的经验驱动的语言变化的程度。研究人员还将查询美国手语的词汇数据库，以测试有关现代ASL的感知形式的预测，并与聋哑人和ASL的听力用户进行行为研究，以测试有关视觉注意力分配的假设，这是由于聋哑和手术的掌握而导致的。在此过程中，研究将提供有关人脑如何更改我们使用的工具（在这种情况下，语言）以及这些工具又如何影响人脑功能的方式的宝贵信息。这将提供对语言变化的更复杂的理解，该语言变化阐明了语言与使用它们的人类之间的复杂互动。该奖项反映了NSF的法定使命，并被认为是值得通过基金会的智力优点和更广泛影响的评估评估来获得支持的。

项目成果

期刊论文数量（12）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

BAPose: Bottom-Up Pose Estimation with Disentangled Waterfall Representations

DOI：
10.1109/wacvw58289.2023.00059
发表时间：
2023-01-01
期刊：
2023 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW)
影响因子：
0
作者：
Artacho, Bruno;Savakis, Andreas
通讯作者：
Savakis, Andreas

Sign Language Recognition, Generation, and Translation: An Interdisciplinary Perspective

DOI：
10.1145/3308561.3353774
发表时间：
2019-01-01
期刊：
ASSETS'19: THE 21ST INTERNATIONAL ACM SIGACCESS CONFERENCE ON COMPUTERS AND ACCESSIBILITY
影响因子：
0
作者：
Bragg, Danielle;Koller, Oscar;Morris, Meredith Ringel
通讯作者：
Morris, Meredith Ringel

Sign language experience redistributes attentional resources to the inferior visual field