A Remote Multimodal Learning Environment to Increase Graphical Information Access for Blind and Visually Impaired Students

远程多模式学习环境可增加盲人和视障学生的图形信息访问

基本信息

批准号：
1822800
负责人：
Nicholas Giudice
金额：
$ 74.79万
依托单位：
University of Maine
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2018
资助国家：
美国
起止时间：
2018-08-15 至 2023-07-31
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1822800&HistoricalAwards=false
关键词：
Remote Multimodal Learning Environment Increase

项目摘要

There are many limitations for students who are blind or visually impaired (BVI) in accessing complex STEM graphical information in the classroom or workplace. This longstanding problem arises due to reliance on inaccessible and outdated learning materials, the need for costly specialized devices, and an adherence to an outdated educational service model. To address these issues, this project will investigate the development and evaluation of an innovative remote learning system based on the use of multiple sensory channels to strategically present information from auditory, linguistic, touch, and enhanced visual sensing. The research will focus specifically on the optimization of multimodal information presentation and perception, separating sensory output based on its unique information processing characteristics for conveying different types of stimuli. The first project goal is to increase the quality of STEM instruction for BVI students by determining perceptually motivated learning supports that promote non-visual knowledge acquisition of STEM graphical and spatial information (learning goal). The second project goal is to increase access to graphical and spatial STEM content through creation of an innovative remote multimodal interface for communicating the conceptual meaning of visual information (technology goal). The project outcomes will contribute to theories of non-visual learning and multisensory processing, and a clear translational path to development of more efficient, intuitive, and usable multimodal interfaces for both blind and sighted users. The application of the results will help to address the severe under-representation of BVI individuals in STEM-related disciplines, and the 70% unemployment rate of this demographic, by providing a new, low-cost, and accessible technology platform for communicating non-visual graphical STEM materials. The researchers will answer the following inter-connected questions: 1) What is the best information content to be conveyed by different modal outputs for maximizing perceptual saliency, learnability, interpretation, and representation of STEM graphical materials? Once optimized in the lab, 2) How well does the optimized multimodal learning system perform in a remote deployment environment in conveying graphical STEM materials to BVI learners; and 3) Does the remote learning system increase the level of comprehension of STEM graphical content as compared to traditional BVI instructional methods? Both quantitative and qualitative data about the optimization process and the remote technology system will be collected and analyzed including user response metrics on speed/accuracy, user experience data, and STEM graphical assessment instruments adapted for BVI students. The first phase of the research will investigate multimodal information processing in order to establish best practices for information delivery and non-visual graphical learning efficiency with experiments comparing graphical information presented in different modalities for three core STEM graphical themes: graphs, diagrams, and maps. The second phase of experiments will investigate the remote learning system's efficacy as well as evaluating user performance on graphical STEM learning measures and key usability and satisfaction metrics. This project has the broader goals of increasing independence for BVI learners and other students with or without disabilities who might benefit from a remote multimodal learning environment, and the development of a new tool for supporting large-scale research and assistive technology evaluation with BVI human subjects, thereby dramatically increasing scientists' ability to recruit and work with a much larger population of BVI users than is currently possible from lab-based studies.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

对于在教室或工作场所访问复杂的STEM图形信息时，盲人或视力障碍（BVI）的学生有很多限制。由于依靠无法访问和过时的学习材料，对昂贵的专业设备的需求以及坚持过时的教育服务模型的需求，因此出现了这个长期存在的问题。为了解决这些问题，该项目将根据使用多个感官渠道来策略性地介绍听觉，语言，触摸和增强的视觉传感的信息的创新远程学习系统的开发和评估。这项研究将专门针对多模式信息表现和感知的优化，并根据其独特的信息处理特征来分开感官输出，以传达不同类型的刺激。第一个项目目标是通过确定促进非视觉知识获取STEM图形和空间信息（学习目标）的知识获取的知识知识（学习目标）来提高BVI学生的STEM教学质量。第二个项目目标是通过创建创新的远程多模式界面来扩展对图形和空间词干内容的访问，以传达视觉信息的概念含义（技术目标）。该项目成果将有助于非视觉学习和多感觉处理的理论，以及为盲人和视力用户开发更高效，直观且可用的多模式接口的明确翻译道路。结果的应用将有助于解决BVI个体在与STEM相关学科中的严重代表性不足，以及该人群中的70％的失业率，通过提供一个新的，低成本且可访问的技术平台，以传达非视图图形的茎材料。研究人员将回答以下相互联系的问题：1）不同模态输出所传达的最佳信息内容是什么，以最大程度地提高知觉显着性，可学习性，解释和STEM图形材料的表示？一旦在实验室中进行了优化，2）优化的多模式学习系统在远程部署环境中的表现如何，以将图形茎材料传达给BVI学习者； 3）与传统的BVI教学方法相比，远程学习系统是否会提高STEM图形内容的理解水平？将收集和分析有关优化过程和远程技术系统的定量和定性数据，包括有关速度/准确性，用户体验数据以及适用于BVI学生的STEM图形评估工具的用户响应指标。该研究的第一阶段将研究多模式信息处理，以建立信息传递和非视觉图形学习效率的最佳实践，并比较以不同方式显示的三个核心STEM图形主题以不同方式显示的图形信息：图形，图表和地图。实验的第二阶段将研究远程学习系统的功效，并评估用户在图形词干学习指标以及关键的可用性和满意度指标上的绩效。该项目具有更广泛的目标，即增加可能从远程多式模式学习环境中受益的BVI学习者和其他有残疾学生的独立性，并开发一种新工具，以支持大规模研究和BVI人类受试者的大规模研究和辅助技术评估，从而大大增加与BVI的大量研究的能力，从而使BVI较大的人群的能力大大增加，从而使BVI较大的人群的能力与更多的研究相比。并被认为是通过基金会的知识分子优点和更广泛的影响审查标准来评估值得支持的。

项目成果

期刊论文数量（5）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

Design Guidelines for Schematizing and Rendering Haptically Perceivable Graphical Elements on Touchscreen Devices

在触摸屏设备上绘制和渲染触觉感知图形元素的设计指南

DOI：
10.1080/10447318.2020.1752464
发表时间：
2020
期刊：
International Journal of Human–Computer Interaction
影响因子：
0
作者：
Palani, Hari P.;Fink, Paul D.;Giudice, Nicholas A.
通讯作者：
Giudice, Nicholas A.

EMBEDDING EXPERT KNOWLEDGE: A CASE STUDY ON DEVELOPING AN ACCESSIBLE DIAGRAMMATIC INTERFACE

嵌入专家知识：开发可访问的图形界面的案例研究

DOI：
发表时间：
2021
期刊：
Psychology of Mathematics Education
影响因子：
0
作者：
Doore, S.A.;Justin K. Dimmel, J.K.;Xi, R.;Giudice, N.A.
通讯作者：
Giudice, N.A.

Comparing Natural Language and Vibro-Audio Modalities for Inclusive STEM Learning with Blind and Low Vision Users

比较自然语言和振动音频模式对盲人和低视力用户进行包容性 STEM 学习

DOI：
10.1145/3597638.3608429
发表时间：
2023
期刊：
In The 25th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’23
影响因子：
0
作者：
Brown, Justin R.;Doore, Stacy A.;Dimmel, Justin K.;Giudice, Norbert;Giudice, Nicholas A.
通讯作者：
Giudice, Nicholas A.

Multimodality as universality: Designing inclusive accessibility to graphical information