CSR: Medium: Improving the Interface between Machine Learning and Software Systems

CSR：中：改进机器学习和软件系统之间的接口

• 批准号: 2313190
• 负责人: Shan Lu
• 金额: $ 60万
• 依托单位: University of Chicago
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-10-01 至 2027-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2313190&HistoricalAwards=false
• 关键词: CSR; Medium; Improving; Interface; between

Machine-Learning-as-a-Service (MLaaS) is a new software paradigm that gives developers access to powerful machine learning models trained on massive data sets without requiring those developers know how to train the models or have access to the required training data. This approach makes machine learning accessible to a much wider range of software systems, but it also creates new challenges. Specifically, there is a tension between the desire to provide a very general MLaaS interface (to make it as widely applicable as possible) and the specific needs of individual applications that use MLaaS. For example, many MLaaS providers offer general object detection models as a service, which can recognize tens of thousands of different objects in a picture, but most applications require only a small subset of that capability; e.g., applications concerned with traffic only care about objects that could appear on a roadway. This project will explore this tension – to preserve the generality of MLaaS while improving the robustness, accuracy, and performance of individual applications that use these services. Specifically, the project will first create a benchmark suite of real-world applications to drive an empirical study of the software bugs that arise due to the tension between general MLaaS interfaces and specific application needs. Based on that study, the project will create a set of tools that automatically adapt software to fix inconsistencies and ambiguities that arise due to the use of general MLaaS interfaces in application-specific contexts. Finally, the project will create methods and tools for refactoring software to use additional information –including the MLaaS’s confidence in its results– that is available from MLaaS providers, but is typically ignored by software applications.Machine learning is now a major part of software systems that affect our daily lives, including transportation, medical systems, and even news distribution. The rise of MLaaS makes it even easier for non-experts to incorporate machine learning into these software systems, but it also increases the opportunities for a new class of software bugs and software failures. This project will identify and categorize the novel class of bugs that can arise from the use of MLaaS in larger software systems and create tools and methodologies to identify and fix those bugs. All benchmarks, data, and software tools developed through this project will be released as open source so that the larger community can freely benefit from this work. By improving the correctness and performance of software systems that use machine learning services, this project will not only make it easier to develop such software, but also improve the quality of people’s daily lives as the software they use will be more reliable.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

机器学习-AS-A-Service（MLAAS）是一种新的软件范式，可让开发人员访问在大规模数据集上训练强大的机器学习模型，而无需这些开发人员知道如何培训模型或可以访问所需的培训数据。这种方法使机器学习可用于更广泛的软件系统，但也带来了新的挑战。具体而言，希望提供非常通用的MLAA接口（以使其尽可能广泛地适用）与使用MLAAS的单个应用程序的特定需求之间存在张力。例如，许多MLAA提供商提供一般对象检测模型作为服务，可以识别图片中数以万计的对象，但是大多数应用程序仅需要该功能的一小部分。例如，仅与交通有关的申请仅关心可能出现在道路上的物体​​。项目将探索这种张力 - 以保持MLAA的一般性，同时提高使用这些服务的单个应用程序的鲁棒性，准确性和性能。具体而言，该项目将首先创建一套现实世界应用程序的基准套件，以推动由于一般MLAAS界面与特定应用程序需求之间的张力而引起的软件错误的经验研究。基于该研究，该项目将创建一组工具，这些工具可以自动调整软件以修复由于在特定于应用程序特定的上下文中使用一般MLAAS界面所致的矛盾和歧义。最后，该项目将创建用于重构软件使用其他信息的方法和工具，包括MLAAS对其结果的信心 - 可从MLAAS提供商获得，但通常被软件应用程序所忽略。MachineLearning现在是影响我们日常生活的软件系统的重要组成部分，包括我们的日常生活，包括运输，医疗系统和新闻分发。 MLAA的兴起使非专家更容易将机器学习纳入这些软件系统，但它也增加了新的软件错误和软件故障的机会。该项目将识别并分类新的错误类别的错误类别，这些错误可能是由于在较大的软件系统中使用MLAA而引起的，并创建工具和方法来识别和修复这些错误。通过该项目开发的所有基准，数据和软件工具都将作为开源发布，以便较大的社区可以从这项工作中自由受益。通过改善使用机器学习服务的软件系统的正确性和性能，该项目不仅可以使开发此类软件更容易，而且还可以提高人们日常生活的质量，因为他们使用的软件将更加可靠。该奖项反映了NSF的法定任务，并通过使用该基金会的知识分子和更广泛的影响来评估Criteria诚实地认为，通过评估诚实的支持。