CHS: Small: Improving Web Accessibility Through Multi-Resolution Mixed-Initiative Interaction Tools

CHS：小型：通过多分辨率混合主动交互工具提高 Web 可访问性

• 批准号: 2007857
• 负责人: Steve Oney
• 金额: $ 49.92万
• 依托单位: Regents of the University of Michigan - Ann Arbor
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2007857&HistoricalAwards=false
• 关键词: CHS; Small; Improving; Web; Accessibility

This research will create powerful new ways to bridge critical accessibility gaps that people with visual impairments face on the Web every day, enabling greater independence and access to information. Many websites are difficult or impossible for users with visual impairments to access through screen readers. Many of these accessibility barriers have been shown currently to be insurmountable without help from a sighted colleague. This project envisions a future in which the Web can be made more navigable and accessible over time while using minimal human effort to bridge access gaps the first time they are encountered, and then automating this bridging support for future interactions. This is done through a mixed-initiative interaction between end users, web automation tools, and remote human workers, at multiple levels of interaction resolution. This allows development of a system that will not only work every time for end users (even on web pages the system has never seen before), but also to leverage the common design elements on the Web more effectively in order to reduce the total amount of human effort needed to successfully learn a new task, significantly reducing cost and improving scalability.In the approach to be developed by this project, when Web users with visual impairments encounter an inaccessible site, they will be able to specify their high-level intent in natural language and then a combination of automated macros and remote helpers will help them achieve their task. Not only do websites differ greatly, but websites often change their underlying structure, and users' goals change as well. Web automation scripts are often unable to infer how to achieve the end user's high-level intent under such dynamic conditions. Human crowd workers, on the other hand, are adept at figuring out which actions to take to achieve a given goal on a website. This work will address some of the fundamental challenges of using automation by employing humans to fill in the gaps where automation fails, and to understand when semantically-meaningful changes have occurred in how a page works. The research is divided into two phases: the first phase focuses on developing methods for creating macros, and the second phase will focus on ways to leverage crowd helpers to make these macros more robust. Both phases will advance the state of the art for Web automation. Progressively the system will be evaluated through a set of laboratory and field studies.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

这项研究将创造有力的新方法，以弥合具有视觉障碍的人每天在网络上面临的关键可访问性差距，从而实现更大的独立性和获取信息。许多网站对于视觉障碍的用户很难或不可能通过屏幕阅读器访问。这些可及性障碍中的许多目前被证明是在没有视力同事的帮助的情况下是无法克服的。该项目设想了一个未来，可以使网络随着时间的流逝而变得更加可通道，可以在第一次遇到的人类努力时弥合访问差距，然后自动化这种桥接支持以对将来的互动进行自动化。这是通过多个互动分辨率的最终用户，Web自动化工具和远程人工工人之间的混合互动来完成的。这允许开发一个系统，该系统不仅可以为最终用户（即使在网页上也从未见过），而且还可以更有效地利用网络上的常见设计元素，以减少成功学习新任务，大大降低成本和改进的可扩展性的方法，以使其在该项目中开发出可比性的范围，从而使他们能够在此范围内开发出高度的范围。自然语言的意图，然后是自动宏和远程助手的组合，将有助于他们完成任务。网站不仅差异很大，而且网站通常会改变其基本结构，并且用户的目标也会改变。 Web自动化脚本通常无法推断如何在这种动态条件下实现最终用户的高级意图。另一方面，人类人群工人擅长弄清楚在网站上实现给定目标的采取行动。这项工作将解决使用自动化来填补自动化失败的空白的一些基本挑战，并了解何时在页面的工作方式中发生语义上的变化。该研究分为两个阶段：第一阶段的重点是开发创建宏的方法，第二阶段将着重于利用人群帮助者使这些宏更强的方法。这两个阶段都将推进Web自动化的最新技术。该系统将通过一组实验室和现场研究进行评估。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子优点和更广泛的影响审查标准通过评估来支持的。

项目成果

ParamMacros: Creating UI Automation Leveraging End-User Natural Language Parameterization

ParamMacros：利用最终用户自然语言参数化创建 UI 自动化

• DOI: 10.1109/vl/hcc53370.2022.9833005
• 发表时间: 2022
• 期刊: 2022 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC
• 作者: Krosnick, Rebecca;Oney, Steve
• 通讯作者: Oney, Steve

Understanding the Challenges and Needs of Programmers Writing Web Automation Scripts

了解程序员编写 Web 自动化脚本的挑战和需求

• DOI: 10.1109/vl/hcc51201.2021.9576476
• 发表时间: 2021
• 期刊: 2021 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC
• 作者: Krosnick, Rebecca;Oney, Steve
• 通讯作者: Oney, Steve