Evaluation-Based Effectiveness Testing

基于评估的有效性测试

• 批准号: 7394619
• 负责人: William B. Hansen
• 金额: $ 15.69万
• 依托单位: TANGLEWOOD RESEARCH, INC.
• 依托单位国家: 美国
• 财政年份: 2008
• 资助国家: 美国
• 起止时间: 2008-07-15 至 2010-01-30
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/7394619
• 关键词: Adopted; Archives; Attenuated; Behavioral; Benchmarking; Code; Communities; Community Services; Control Groups; Data; Data Collection; Data Reporting; Data Set; Databases; Development; Education; Effectiveness; Ensure; Evaluation; Funding; Goals; Gold; Intervention; Lizards; Measurement; Measures; Mediator of activation protein; Methods; Online Systems; Outcome; Paper; Participant; Performance; Phase; Policy Maker; Prevention; Prevention program; Program Effectiveness; Program Evaluation; Provider; Randomized Controlled Trials; Registries; Research; Research Personnel; Resources; Sampling; Schools; Services; Small Business Funding Mechanisms; Small Business Innovation Research Grant; Standards of Weights and Measures; Statistical Methods; Sum; Surveys; System; Testing; United States Substance Abuse and Mental Health Services Administration; base; comparison group; design; drug abuse prevention; experience; programs; tool

DESCRIPTION (provided by applicant): Schools, communities, and service providers are beginning to implement programs with proven or promising potential. As part of this dissemination effort, local implementers are often required to conduct local evaluations. These evaluations, however, are often underpowered, lack appropriate or any comparison group, and are subject to attenuated effectiveness estimates due to unreliable measures or sample invariance. The goal of this SBIR is to develop a tool to help local evaluators and policy makers obtain valid, reliable, and robust evidence for the effectiveness of their intervention practices without the burden of having to identify convene, maintain, and measure a local comparison group. The system will meta-analytically combine archived data from prior evaluations, including evaluation data from intervention groups of relevant programs and from control groups to establish appropriate counterfactuals for local evaluators. Strategies will be developed that will allow users to complete analyses that will useful for interpreting local evaluation outcomes. Thus, rather than relying on a typically unobtainable gold-standard method for assessing impact (randomized control trials), the goal of this approach will be to provide a statistically reliable method for determining local program performance and to provide programs useful benchmarks against which local evaluation findings can be interpreted. This project will take advantage of an existing resource, previously developed with SBIR funds that assists schools and communities in performing local evaluations. This existing system, Evaluation Lizard, is now an active service and has, during the past several years amassed a large database that includes not only survey data on behavioral outcomes, mediators related to outcomes, and demographic information about survey participants, but also ties these data to programs being implemented and the evaluation design that was adopted. In this Phase I project we will: 1) We will assemble extant datasets that are appropriate for completing a pilot test of proposed methods and ensure that data are coded to provide appropriate pools of results for completing meta-analytic analyses; 2) refine meta-analytic statistical methods for completing comparisons using referent data; and 3) test these methods using data from five to ten local evaluations to develop and then demonstrate the utility of the proposed methods for local program evaluators, service providers, and funders.

描述（由申请人提供）：学校、社区和服务提供商正在开始实施已证明或有希望的潜力的计划。作为传播工作的一部分，当地实施者通常需要进行当地评估。然而，这些评估往往力度不足，缺乏适当的或任何比较组，并且由于不可靠的测量或样本不变性而受到有效性估计的削弱。该 SBIR 的目标是开发一种工具，帮助当地评估者和政策制定者获得有效、可靠和有力的证据，证明其干预实践的有效性，而无需确定召集、维持和衡量当地比较组。该系统将以荟萃分析的方式结合先前评估的存档数据，包括来自相关项目干预组和对照组的评估数据，以便为当地评估者建立适当的反事实。将制定策略，允许用户完成有助于解释当地评估结果的分析。因此，该方法的目标不是依赖通常无法获得的金标准方法来评估影响（随机对照试验），而是提供一种统计上可靠的方法来确定当地项目绩效，并为当地评估提供有用的基准。研究结果是可以解释的。该项目将利用先前由 SBIR 资金开发的现有资源，帮助学校和社区进行当地评估。这个现有的系统，Evaluation Lizard，现在是一项活跃的服务，在过去几年中积累了一个大型数据库，其中不仅包括行为结果的调查数据、与结果相关的中介因素以及调查参与者的人口统计信息，而且还将这些数据联系起来正在实施的计划和所采用的评价设计的数据。在第一阶段项目中，我们将： 1) 我们将收集适合完成所提出方法的试点测试的现有数据集，并确保对数据进行编码，以提供适当的结果池来完成荟萃分析； 2）完善荟萃分析统计方法，以使用参考数据完成比较； 3) 使用来自五到十个当地评估的数据来测试这些方法，以开发并展示所提出的方法对当地项目评估者、服务提供者和资助者的实用性。

项目成果

A Synthetic Comparator Approach to Local Evaluation of School-Based Substance Use Prevention Programming.

对基于学校的药物使用预防规划进行本地评估的综合比较方法。

• 发表时间: 2014-06
• 影响因子: 2.9
• 作者: Hansen, William B;Derzon, James H;Reese, Eric L
• 通讯作者: Reese, Eric L