HCC: Small: AccessMath: Improving Mathematics Lectures for Low Vision Students through Integrated Video, Note-Taking and Search

HCC：小型：AccessMath：通过集成视频、笔记和搜索改进低视力学生的数学讲座

• 批准号: 1218801
• 负责人: Stephanie Ludi
• 金额: $ 48.68万
• 依托单位: Rochester Institute of Tech
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2012
• 资助国家: 美国
• 起止时间: 2012-09-01 至 2017-03-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1218801&HistoricalAwards=false
• 关键词: HCC; Small; AccessMath; Improving; Mathematics

Access to mathematics education is critical to facilitating careers in the sciences, engineering, technology, and in many professions, as well as to providing the mathematical literacy needed in everyday life. The PI's goal in this project is to increase real-time access in the classroom to mathematical material for low vision students (who are visually impaired but not blind), which is currently mostly lacking as these students must primarily rely on assistive technology and human note takers to provide access to course materials after the class meeting. The PI's approach is to bring together expertise in computer science, mathematics, and accessibility to provide a useful and usable solution to this problem for the iPad. Called AccessMath, the prototype application will provide zoom-able camera views as well as a view of notes written at the whiteboard (captured by a Mimio), which may be altered through magnification and contrast adjustment features. The system will support consulting and searching course content during lecture, including textbooks, handouts, notes written at the whiteboard, and audio in video (e.g., to find the word "eigenvalue" in recorded lectures for the course). A novel note-taking facility will be employed, where notes are represented on note "cards" that can be attached to a specific location in the current view (e.g., at a spot in the whiteboard view), then manually moved, resized, rotated, stacked, and "flicked" out of view, providing a natural means for interacting informally with notes created during lecture. Notes are created as blank cards, or as cards containing views of whiteboard data, video excerpts, or existing notes. Notes are also used in constructing search queries. Linear algebra (in particular matrices) is the pilot scope for the project, due to the many challenges in conveying a matrix, individual items within the matrix, following derivations, and the scale of the matrix needed when completing non-trivial problems. AccessMath users will have a single iPad which connects to a server providing video feeds and recognition and retrieval services.Broader Impacts: AccessMath will assist low-vision students with mathematical instruction in and out of class, at the high school and university levels, by providing immediate access to lecture materials, including searches for mathematical content in whiteboard notes, handouts, textbooks, and lecture videos (through word spotting in audio). The system will afford student ownership in note taking and customizing access, which will be significantly better than current solutions, and will reduce or eliminate the extent to which visually impaired students fall behind their sighted peers in the classroom setting. The project will also advance the state-of-the-art in recognition and retrieval for mathematical notation, and will create a novel note-taking model using the metaphor of photos on a tabletop, which should be applicable to domains other than mathematics.

接受数学教育对于促进科学、工程、技术和许多职业的职业生涯以及提供日常生活所需的数学素养至关重要。 PI 在该项目中的目标是增加低视力学生（视力障碍但不是盲人）在课堂上实时获取数学材料的机会，而目前这方面是最缺乏的，因为这些学生必须主要依赖辅助技术和人工笔记学员在班会后提供课程材料的访问权限。 PI 的方法是汇集计算机科学、数学和可访问性方面的专业知识，为 iPad 的这一问题提供有用且可用的解决方案。 该原型应用程序名为 AccessMath，将提供可缩放的相机视图以及在白板上书写的笔记视图（由 Mimio 捕获），这些视图可以通过放大和对比度调整功能进行更改。 该系统将支持在讲座期间查阅和搜索课程内容，包括教科书、讲义、白板上写的笔记以及视频中的音频（例如，在课程录制的讲座中查找“特征值”一词）。 将采用一种新颖的笔记工具，其中笔记表示在笔记“卡”上，可以将其附加到当前视图中的特定位置（例如，白板视图中的某个位置），然后手动移动、调整大小、旋转，堆叠起来，并“弹开”出视野，提供了一种与讲座期间创建的笔记进行非正式交互的自然方式。 笔记被创建为空白卡片，或包含白板数据、视频摘录或现有笔记视图的卡片。 注释也用于构建搜索查询。 线性代数（特别是矩阵）是该项目的试点范围，因为在传达矩阵、矩阵中的各个项目、以下推导以及完成重要问题时所需的矩阵规模方面存在许多挑战。 AccessMath 用户将拥有一台 iPad，该 iPad 连接到提供视频源以及识别和检索服务的服务器。 更广泛的影响：AccessMath 将帮助低视力学生在高中和大学的课堂内外进行数学教学，方法是：立即访问讲座材料，包括搜索白板笔记、讲义、教科书和讲座视频中的数学内容（通过音频中的单词识别）。 该系统将为学生提供记笔记和定制访问的所有权，这将明显优于当前的解决方案，并将减少或消除视障学生在课堂环境中落后于视力正常的同龄人的程度。 该项目还将推进数学符号识别和检索的最先进水平，并将使用桌面上照片的隐喻创建一种新颖的笔记模型，该模型应该适用于数学以外的领域。