Evaluation-Based Effectiveness Testing

基于评估的有效性测试

• 批准号: 8248162
• 负责人: William B. Hansen
• 金额: $ 48.42万
• 依托单位: TANGLEWOOD RESEARCH, INC.
• 依托单位国家: 美国
• 财政年份: 2011
• 资助国家: 美国
• 起止时间: 2011-06-01 至 2014-05-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/8248162
• 关键词: Algorithms; Archives; Behavioral; Benchmarking; Charge; Client; Communities; Community Services; Control Groups; Data; Data Analyses; Data Reporting; Databases; Development; Drug Prevention Program; Drug usage; Education; Effectiveness; Evaluation; Funding; Goals; Gold; Intervention; Link; Lizards; Mediating; Meta-Analysis; Methods; Online Systems; Outcome; Participant; Performance; Pharmaceutical Preparations; Phase; Policy Maker; Prevention; Prevention program; Program Evaluation; Protocols documentation; Provider; Randomized Controlled Trials; Registries; Relative (related person); Reporting; Research; Research Personnel; Schools; Services; Small Business Innovation Research Grant; Statistical Methods; Sum; Surveys; System; Testing; United States Substance Abuse and Mental Health Services Administration; Work; base; comparison group; design; drug abuse prevention; evidence base; group intervention; innovation; prevention evaluation; programs; public health relevance; reconstitution; tool

DESCRIPTION (provided by applicant): The goal of this SBIR is to develop an online tool, Synthetic Controls, to help local evaluators and others charged with demonstrating evidence of effectiveness of drug prevention programs obtain evidence for assessing the effectiveness of their interventions. Synthetic Controls will provide useful analyses and simultaneously relieve local evaluators of the burden of having to identify, convene, maintain, and administer surveys to a comparison group. The system will meta-analytically combine archived data from previously administered surveys - including evaluation data from intervention groups of relevant programs and from control groups where available - to establish counterfactuals for local evaluators. Thus, rather than relying on gold-standard methods (e.g. randomized control trials) for assessing impact, the goal of this approach will be to provide a statistically reliable method for determining the impact of local programs by comparing current performance against benchmarks set by an accumulated database of previous evaluations. During Phase I, we were able to demonstrate the feasibility of using an extant database for completing a pilot test of proposed methods. We refined statistical methods for completing analyses using referent data. We completed extensive analyses to demonstrate the feasibility and utility of the proposed methods. During Phase II, we propose to complete the following tasks: 1. We will design a database that will efficiently reconstitute various configurations of data to allow users to select appropriate comparison groups. 2. We will develop data analysis algorithms that will construct appropriate comparisons for behavioral and mediating variable analyses. 3. We will develop protocols to automatically link the application to data collected by local evaluations conducted using Evaluation Lizard. 4. We will develop protocols to provide a method to import data collected by local evaluations conducted using other (not Evaluation Lizard) services. 5. We will design a Synthetic Controls user interface that will allow clients to register, manage projects, complete analyses, and download reports. 6. We will conduct an alpha test to debug the system. 7. We will conduct a beta test to demonstrate that the system will work with a variety of potential clients. PUBLIC HEALTH RELEVANCE: Currently, local evaluations of drug prevention programs lack the ability to assess effectiveness because few, if any, include control or comparison groups against which to establish the degree to which they have reduced the rate of drug use onset. This project will provide an innovative strategy whereby local drug prevention evaluations can compare their findings with the findings of previous local evaluations. This will allow programs to assess their relative effectiveness.

描述（由申请人提供）：本 SBIR 的目标是开发一种在线工具“综合控制”，以帮助当地评估人员和其他负责证明毒品预防计划有效性证据的人员获得评估其干预措施有效性的证据。综合对照将提供有用的分析，同时减轻当地评估人员必须确定、召集、维护和管理对照组调查的负担。该系统将以荟萃分析的方式结合以前进行的调查的存档数据（包括来自相关项目的干预组和可用的对照组的评估数据），为当地评估者建立反事实。因此，该方法的目标不是依靠黄金标准方法（例如随机对照试验）来评估影响，而是通过将当前绩效与累积的绩效设定的基准进行比较，提供统计上可靠的方法来确定当地计划的影响。以前的评估数据库。 在第一阶段，我们能够证明使用现有数据库完成所提出方法的试点测试的可行性。我们改进了使用参考数据完成分析的统计方法。我们完成了广泛的分析，以证明所提出方法的可行性和实用性。 在第二阶段，我们建议完成以下任务： 1. 我们将设计一个数据库，该数据库将有效地重构各种数据配置，以允许用户选择合适的比较组。 2. 我们将开发数据分析算法，为行为和中介变量分析构建适当的比较。 3. 我们将开发协议，自动将应用程序链接到使用Evaluation Lizard 进行的本地评估收集的数据。 4. 我们将制定协议，提供一种方法来导入使用其他（非评估蜥蜴）服务进行的本地评估收集的数据。 5. 我们将设计一个综合控制用户界面，允许客户注册、管理项目、完成分析和下载报告。 6. 我们将进行alpha测试来调试系统。 7. 我们将进行 Beta 测试，以证明该系统可以与各种潜在客户配合使用。 公共卫生相关性：目前，当地对毒品预防计划的评估缺乏评估有效性的能力，因为很少有（如果有的话）包括对照组或比较组来确定其降低吸毒发病率的程度。该项目将提供一项创新战略，使地方毒品预防评估能够将其结果与以前的地方评估结果进行比较。这将使计划能够评估其相对有效性。