SoD-HCER: Evaluation of Complex Designs--A Comparative Study

SoD-HCER：复杂设计的评估——比较研究

• 批准号: 0613601
• 负责人: Michael Hicks
• 金额: --
• 依托单位: University of Maryland, College Park
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2006
• 资助国家: 美国
• 起止时间: 2006-08-01 至 2008-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0613601&HistoricalAwards=false
• 关键词: SoD; HCER; Evaluation; Complex; Designs

A critical requirement for any science is the ability to measure. Without measurement, it is difficult to evaluate and improve. Not only is our ability to measure and evaluate software designs today weak, we also lack a thorough understanding of the shortcomings of the strengths and weaknesses of existing software design evaluation methods. To learn more about the shortcomings of current evaluation methods, the PI will in this project study two software products from the systems domain which share the important design criteria of modifiability and performance, the Click modular router and the Jetty http server and servlet container. For each of these two programs, there exist clear design goals and extensive change histories. Using change history data, the PI will create an approximate "master list" of modifiability and performance flaws in early versions of each program. For example, a feature request that required changes to many files could be flagged as a modifiability flaw. She will then apply several evaluation methods (e.g., quantitative software metrics, architecture-based evaluation, and semiautomatic methods such as design snippets) to early versions of each subject program, and compare the output of each method to the approximate master list of design flaws. Project outcomes will include establishment of a benchmark and experimental protocol that can be used to critique new techniques or tools for evaluating software designs using change history logs, and an increased understanding of the deficiencies of existing techniques and tools for evaluating software designs.Broader Impacts: Software designs matter because software matters. To improve software designs, we must have adequate tools for evaluating candidate designs. This project will provide insight into new tools and techniques for software design evaluation. Programmers using these new tools and techniques will produce better software designs and thus better software. In addition, the benchmark produced by this experiment will prove valuable to software engineering educators; instructors will be able to design assignments and projects that involve the benchmark, and which will expose students to issues in evaluating software designs and empirical software engineering research.

任何科学的关键要求是衡量能力。 没有测量，很难评估和改进。 我们当今的衡量和评估软件设计的能力不仅弱，我们还缺乏对现有软件设计评估方法优势和劣势的缺点的彻底理解。 为了了解有关当前评估方法缺点的更多信息，PI将在该项目研究中从系统域中研究两个软件产品，这些软件产品具有可修改性和性能的重要设计标准，即单击模块化路由器以及Jetty HTTP Server和Servlet容器。 对于这两个程序中的每一个，都有明确的设计目标和广泛的变化历史。 使用更改历史记录数据，PI将在每个程序的早期版本中创建一个近似的可修改性和性能缺陷的“主列表”。 例如，需要更改许多文件的功能请求可以标记为可修改性缺陷。 然后，她将在每个主题程序的早期版本中应用多种评估方法（例如，基于架构的定量软件指标，基于体系结构的评估和半自动方法（例如设计摘要）），并将每种方法的输出与设计缺陷的近似主列表进行比较。 项目成果将包括建立基准和实验协议，该协议可用于批评使用变更历史日志评估软件设计的新技术或工具，以及对现有技术的缺陷和用于评估软件设计的工具的不足的了解。软件设计很重要，因此软件设计很重要。 为了改善软件设计，我们必须拥有足够的工具来评估候选设计。 该项目将洞悉用于软件设计评估的新工具和技术。 使用这些新工具和技术的程序员将生成更好的软件设计，从而更好地软件。 此外，该实验产生的基准将证明对软件工程教育者很有价值；讲师将能够设计涉及基准测试的任务和项目，并将学生遇到评估软件设计和经验软件工程研究的问题。