Measuring the Quality of Delivery of Prevention

衡量预防的质量

• 批准号: 7909777
• 负责人: William B. Hansen
• 金额: $ 43.18万
• 依托单位: TANGLEWOOD RESEARCH, INC.
• 依托单位国家: 美国
• 财政年份: 2007
• 资助国家: 美国
• 起止时间: 2007-09-30 至 2012-04-30
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/7909777
• 关键词: Adherence; Administrator; Adopted; Affect; Area; Awareness; Back; Bar Codes; Benchmarking; Collection; Communities; Congresses; Data; Data Collection; Databases; Development; Dimensions; Educational process of instructing; Effectiveness; Elements; Ensure; Environment; Evaluation; Evidence based program; Failure; Funding; Future; Generations; Goals; Government Agencies; In Situ; Indium; Intervention; Interview; Link; Lizards; Manuals; Measures; Methods; Modeling; Monitor; Online Systems; Outcome; Participant; Performance; Phase; Policy Maker; Prevention; Prevention Research; Prevention program; Printing; Process; Program Evaluation; Provider; Qualifying; Reporting; Research; Research Personnel; Schools; Small Business Innovation Research Grant; Standardization; Students; Surveys; System; Testing; Training; Validity and Reliability; Video Recording; Work; base; design; dosage; drug abuse prevention; federal policy; improved; meetings; phase 2 study; prevention evaluation; programs; prototype; public health relevance; satisfaction; standardize measure; success; teacher; tool; wasting

DESCRIPTION (provided by applicant): Quality of delivery is known to moderate the effectiveness of drug abuse prevention programs. As research-based programs become widely disseminated, having tools for assessing quality of delivery becomes important for documenting and understanding the conditions under which programs succeed and fail. There are currently no standard methods for measuring the quality with which prevention programs are delivered. The goal of this project is to create a standardized tool for assessing quality of delivery for research-based drug abuse prevention programs.. Commercially, the addition of this product to an existing product (Evaluation Lizard) that provides pretest and posttest surveys tailored to the evaluation of evidence-based programs, will significantly strengthen our niche in the prevention evaluation marketplace. The broad aim of this SBIR project is to develop a tool that will allow the collection of standardized measures of quality of delivery across a wide range of programs being disseminated as a result of their inclusion on the NREPP model programs list and newly developed programs that wish to qualify for future inclusion. During Phase I, we interviewed prevention researchers, program administrators, program developers, and policy makers to define and elaborate on the essential criteria for defining dimensions of quality of implementation. Based on their input, we designed a prototype system that allowed us to accomplish the following tasks: (1) create session and program rating form templates for selected NREPP facilitator-delivered prevention programs, (2) link templates to a database to track form information (program, session, observer, teacher, and date of creation), and (3) create PDF forms printed remotely that can be linked by bar code back to the database. The prototype system measures dosage, adherence, engagement, and adaptation. We also developed a manual for using the system. While functional for the pilot test, this system is not yet fully automated and does not include numerous elements. During Phase II, our first task will be to complete the quality of delivery data gathering system for collecting quality of delivery data that will include tools for assessing adherence, adaptation, dosage, engagement, and measures of overall quality. We will develop online applications that will allow data collected about quality of program delivery to be linked and jointly analyzed in various ways with pretest-posttest outcome data from students. To ensure that the application works as intended, we will conduct alpha and beta tests of all component applications. We will develop an online system for conducting basic analysis and preparing reports that meets the needs of five separate professional groups: program implementers, program administrators, program developers, prevention researchers, and policy makers. We will conduct a field trial with 10 prevention agencies and school districts to demonstrate that the quality of delivery data collection and reporting system provide program implementers, prevention agency administrators and supervisors, and government agency prevention administrators with information useful to documenting and understanding the process of prevention program delivery. Established prevention agencies will adopt the use of the Evaluation Lizard system, including both pretest-posttest surveys and quality of delivery report forms. We hypothesize that participating groups will report satisfaction with the functioning of the system and will report increased awareness of how prevention is being implemented in their area of purview. Further, we will conduct a reliability and validity sub-study in which participating agencies will be asked to select one teacher to video record the lessons they teach so that observation data can be included in the analysis in order to demonstrate reliability and validity of the measures. PUBLIC HEALTH RELEVANCE: The field of drug abuse prevention research has identified numerous interventions that can be effective; however, unless programs are implemented well in practice, they are unlikely to achieve their desired effect, and the effort and expense of these programs may be wasted. The need to assess quality of implementation is widely recognized, but, to date, there is no unified strategy for gathering or reporting about data. The value of the product we are proposing is that it would provide an easy-to-use system for documenting quality of program implementation that could be linked to outcome data.

描述（由申请人提供）：众所周知，交付质量会调节药物滥用预防计划的有效性。随着基于研究的项目得到广泛传播，拥有评估交付质量的工具对于记录和理解项目成功和失败的条件变得非常重要。目前没有标准方法来衡量预防计划实施的质量。该项目的目标是创建一个标准化工具，用于评估基于研究的药物滥用预防计划的交付质量。在商业上，将该产品添加到现有产品（Evaluation Lizard）中，该产品提供针对药物滥用量身定制的前测和后测调查。对基于证据的计划的评估将显着增强我们在预防评估市场中的地位。该 SBIR 项目的总体目标是开发一种工具，允许收集各种项目的交付质量标准化衡量标准，这些项目因包含在 NREPP 示范项目清单中而被传播，以及新开发的项目希望以获得未来纳入的资格。在第一阶段，我们采访了预防研究人员、项目管理人员、项目开发人员和政策制定者，以定义和详细说明定义实施质量维度的基本标准。根据他们的意见，我们设计了一个原型系统，使我们能够完成以下任务：(1) 为选定的 NREPP 协调员提供的预防计划创建会话和计划评级表格模板，(2) 将模板链接到数据库以跟踪表格信息（节目、会议、观察员、教师和创建日期），以及 (3) 创建远程打印的 PDF 表格，该表格可以通过条形码链接回数据库。该原型系统测量剂量、依从性、参与度和适应度。我们还开发了该系统的使用手册。虽然可以进行试点测试，但该系统尚未完全自动化，并且不包含众多元素。在第二阶段，我们的首要任务是完成交付质量数据收集系统，用于收集交付质量数据，其中包括评估依从性、适应、剂量、参与度和整体质量衡量的工具。我们将开发在线应用程序，使收集到的有关课程交付质量的数据能够以各种方式与学生的前测后测结果数据进行链接和联合分析。为了确保应用程序按预期工作，我们将对所有组件应用程序进行 alpha 和 beta 测试。我们将开发一个在线系统，用于进行基本分析和准备报告，以满足五个不同专业群体的需求：项目实施者、项目管理员、项目开发者、预防研究人员和政策制定者。我们将与 10 个预防机构和学区进行现场试验，以证明交付质量数据收集和报告系统为计划实施者、预防机构管理者和监督者以及政府机构预防管理者提供有用的信息，以记录和理解交付过程。预防计划的交付。已建立的预防机构将采用评估蜥蜴系统，包括前测后测调查和交付质量报告表。我们假设参与团体将对系统的运作表示满意，并将报告对如何在其职权范围内实施预防的认识有所提高。此外，我们将进行一项信度和效度子研究，要求参与机构选择一名老师，对他们所教的课程进行视频录制，以便将观察数据纳入分析中，以证明措施的信度和效度。 公共卫生相关性：药物滥用预防研究领域已经确定了许多有效的干预措施；然而，除非在实践中很好地实施这些方案，否则它们不可能达到预期的效果，并且这些方案的努力和费用可能会被浪费。人们广泛认识到评估实施质量的必要性，但迄今为止，还没有收集或报告数据的统一策略。我们提议的产品的价值在于，它将提供一个易于使用的系统，用于记录可以与结果数据相关联的计划实施的质量。