Generating Accurate Estimates of Required Sample Size for Multilevel Implementation Studies in Mental Health

生成心理健康多层次实施研究所需样本量的准确估计

• 批准号: 10370396
• 负责人: Nathaniel J. Williams
• 金额: $ 11.76万
• 依托单位: BOISE STATE UNIVERSITY
• 依托单位国家: 美国
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-03-11 至 2024-02-29
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/10370396
• 关键词: Accounting; Address; Adoption; American; Caring; Characteristics; Clinical; Communities; Data Analyses; Data Set; Foundations; Funding; Future; Health; Individual; Intervention; Literature; Manuscripts; Measurement; Mental Health; Mental disorders; Methodology; Methods; Mission; National Institute of Mental Health; Outcome; Partner in relationship; Patient Care; Patient-Focused Outcomes; Patients; Personal Satisfaction; Pilot Projects; Population; Predictive Factor; Process; Provider; Publishing; Reference Values; Reporter; Research; Research Design; Research Personnel; Resources; Risk; Sample Size; Sampling; Science; Scientist; Speed; System; Testing; Translations; United States National Institutes of Health; Work; behavioral health; community setting; cost; design; evidence base; experience; health care quality; health care settings; implementation design; implementation determinants; implementation research; implementation science; implementation strategy; implementation study; improved; interest; patient population; power analysis; predictive modeling; prevent; study characteristics; study population; tool

Project Summary This project fills a major methods gap that prevents investigators from designing studies with accurate esti- mates of required sample size for multilevel behavioral health implementation studies. Implementation science is essential to achieving NIMH’s mission. An essential step in designing implementation studies is to conduct a statistical power analysis to determine the minimum sample size required to statistically detect effects of interest. Power analyses for implementation research are more complicated because they need to account for (a) patients nested within providers who are nested within organizations or other systems, and (b) scientific aims that typically focus on testing (or at a minimum accounting for) cross-level effects of higher-level (e.g., organization, clinician) implementation determinants or strategies on lower-level (e.g., patient) outcomes. While multilevel power anal- ysis tools are available to accommodate these types of nested studies, the tools require investigators to have prior estimates of three key design parameters to determine the proper sample size for their study —intraclass correlation coefficient (ICC), effect size, and proportion of variance explained by covariates—which are not rou- tinely available from the published literature and cannot be reliably estimated from small pilot studies. Power analyses that use inaccurate estimates of these design parameters are highly likely to be either underpowered, and consequently at-risk of not detecting important effects, or over-powered, and consequently wasteful of lim- ited resources. Lack of reference values for these parameters is a foundational barrier to the field because even small changes in design parameters can dramatically alter the effective sample size from N=300 to N=50. NIMH has funded a large number of implementation studies during the last 10 years (N=140) which provides an opportunity for us to re-access the datasets from these projects to generate accurate estimates of multilevel design parameters for behavioral health implementation studies. We will use NIH-RePorter to identify all NIMH- funded behavioral health implementation studies conducted during the last 10 years and collaborate with PIs to extract design parameters for targeted implementation and clinical outcomes, which we will summarize and pub- lish for the field. We will also generate a predictive model that enables PIs to estimate design parameters tailored to the characteristics of their new studies. Building on our preliminary work within the Penn NIMH ALACRITY Center, this project will (1) generate pooled estimates and ranges of design parameters (i.e., ICCs, effect sizes, covariate R2) needed to accurately estimate sample size in multilevel behavioral health implementation studies, and (2) identify the study characteristics that predict the magnitude of these design parameters. Completion of this work will remove a ubiquitous methodological barrier that undermines the advancement of implementation science in behavioral health. The study will contribute to higher quality, more replicable science, more efficient use of NIMH resources, and higher impact implementation research to improve healthcare quality and well-being for millions of individuals who experience psychiatric disorders each year.

项目概要 该项目填补了一个主要的方法空白，该空白阻碍了研究人员设计具有准确估计的研究 多层次行为健康实施研究所需样本量的配合。 对于实现 NIMH 的使命至关重要，设计实施研究的一个重要步骤是进行一项研究。 统计功效分析以确定密切检测感兴趣的影响所需的最小样本量。 研究的功效分析实施更加复杂，因为它们需要考虑 (a) 患者 嵌套在组织或其他系统中的提供者中，以及（b）通常具有的科学目标 专注于测试（或至少考虑）更高级别（例如组织、临床医生）的跨级别影响 低水平（例如患者）结果的实施决定因素或策略。 ysis 工具可用于适应这些类型的嵌套研究，这些工具要求研究人员 事先估计三个关键设计参数，以确定其研究的适当样本量——组内 相关系数（ICC）、效应大小和协变量解释的方差比例——这些不是鲁棒性的 可以从已发表的文献中获得，并且无法通过小型试点研究进行可靠估计。 使用这些设计参数的不准确估计进行的分析很可能是动力不足， 因此存在无法检测到重要影响或过度强大的风险，从而浪费了有限的时间 缺乏这些参数的参考值是该领域的基本障碍，因为即使 设计参数的微小变化可以显着地将有效样本量从 N=300 更改为 N=50。 NIMH 在过去 10 年中资助了大量实施研究（N=140），其中提供了 我们有机会重新访问这些项目的数据集，以生成准确的多层次估计 我们将使用 NIH-RePorter 来识别所有 NIMH- 的行为健康研究实施的设计参数。 资助过去 10 年进行的行为实施健康研究，并与 PI 合作 提取设计参数以进行有针对性的总结和实施临床结果，我们将并发布这些参数 我们还将生成一个预测模型，使 PI 能够估计定制的设计参数。 以我们宾夕法尼亚大学 NIMH ALACRITY 的前期工作为基础，了解他们的新研究的特点。 中心，该项目将 (1) 生成设计参数的汇总估计和范围（即 ICC、效应大小、 协变量 R2）需要准确估计多层次行为健康实施研究中的样本量， (2) 确定预测这些设计参数大小的研究特征。 这项工作将消除阻碍实施进展的普遍存在的方法障碍 该研究将有助于提高科学质量、提高可复制性、提高效率。 利用 NIMH 资源和影响力更大的实施研究来提高医疗保健质量和福祉 每年有数百万遭受精神疾病的人。