CAREER: Interactive Program Synthesis for Web Automation

职业：Web 自动化的交互式程序综合

• 批准号: 2236233
• 负责人: Xinyu Wang
• 金额: $ 52.49万
• 依托单位: Regents of the University of Michigan - Ann Arbor
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-03-01 至 2028-02-29
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2236233&HistoricalAwards=false
• 关键词: CAREER; Interactive; Program; Synthesis; Web

Process Automation continues to be a main driver of digital transformation. Yet, it is still technically very demanding to create such automation programs. This project aims to significantly lower the technical barrier of creating web automation programs. The project’s novelties are a suite of new rewrite-based program synthesis algorithms that can automatically generate programs from user demonstrations. The project’s impacts are to enable non-experts in need of performing a tedious but programmatic web-related task to create a program to automate the work, even if they have little or no background in web programming. At the core, this project is developing algorithms that take as input a user demonstration – in the form of a trace A of user actions (e.g., clicking buttons, scraping data) – and synthesize a parameterized program P with control-flow structures by rewriting A to P. The approach is based on finite tree automata and involves both neural and symbolic elements in the underlying web automation language, with the goal of being able to effectively reason about the webpage contents while still leveraging the underlying webpage structure. The project is curating a new suite of web automation tasks that are independently useful for future research beyond this project. The investigator is collaborating with partners at Michigan (e.g., Women in Science and Engineering (WISE), Michigan Louis Stokes Alliance for Minority Participation (MI-LSAMP) and Engineering Center for Academic Success (ECAS)) to increase participation of students from underrepresented groups in his research.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

流程自动化仍然是数字转换的主要驱动力。但是，创建这样的自动化程序在技术上仍然非常要求。该项目旨在大大降低创建Web自动化程序的技术障碍。该项目的新颖性是一套新的基于重写的程序合成算法，可以自动从用户演示中生成程序。该项目的影响是启用非专家，需要执行乏味但程序化的Web相关任务，以创建一个程序以自动化工作，即使它们在Web编程中几乎没有背景。在核心方面，该项目正在开发算法，以输入用户演示的形式 - 用户操作的形式（例如，单击按钮，刮擦数据）的形式（例如，通过将Control-Flow结构的参数化的程序p）综合为通过RETRATION a的参数化progration a的方法。关于网页内容，同时仍在利用基础网页结构。该项目正在策划新的Web自动化任务套件，这些任务对于超越该项目的未来研究非常有用。研究者正在与密歇根州的合作伙伴（例如，科学与工程领域的妇女（明智），密歇根州路易斯·路易斯·斯托克·斯托克·斯托克·斯托克·斯托克·斯托克联盟（MI-LSAMP）（MI-LSAMP）和工程学术成功中心（ECAS））和工程中心（ECAS）），以提高他的研究中的学生的参与，以表现出众多的研究，以反映了NSF的基础，并以deem deem的成立为基础，并以此为基础。影响审查标准。

项目成果

DiLogics: Creating Web Automation Programs with Diverse Logics

• DOI: 10.1145/3586183.3606822
• 发表时间: 2023-08
• 期刊: Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology
• 作者: Kevin Pu;Jim Yang;Angel Yuan;Minyi Ma;Rui Dong;Xinyu Wang;Yuanchun Chen;Tovi Grossman

Efficient Bottom-Up Synthesis for Programs with Local Variables

具有局部变量的程序的高效自下而上综合

• DOI: 10.1145/3632894
• 发表时间: 2024
• 期刊: Proceedings of the ACM on Programming Languages
• 作者: Li, Xiang;Zhou, Xiangyu;Dong, Rui;Zhang, Yihong;Wang, Xinyu
• 通讯作者: Wang, Xinyu

MIWA: Mixed-Initiative Web Automation for Better User Control and Confidence

MIWA：混合主动 Web 自动化可实现更好的用户控制和信心

• DOI: 10.1145/3586183.3606720
• 发表时间: 2023
• 期刊: ACM
• 作者: Chen, Weihao;Liu, Xiaoyu;Zhang, Jiacheng;Lam, Ian Iong;Huang, Zhicheng;Dong, Rui;Wang, Xinyu;Zhang, Tianyi
• 通讯作者: Zhang, Tianyi