Developing user-centric training in rigorous research: post-selection inference, publication bias, and critical evaluation of statistical claims.

在严谨的研究中开展以用户为中心的培训：选择后推断、发表偏见和统计声明的批判性评估。

• 批准号: 10721491
• 负责人: Carl T Bergstrom
• 金额: $ 9.39万
• 依托单位: UNIVERSITY OF WASHINGTON
• 依托单位国家: 美国
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-08-01 至 2026-07-31
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/10721491
• 关键词: Accounting; Address; Area; Attention; Awareness; Biological Process; Biological Sciences; Biomedical Research; Books; Collaborations; Coupled; Data; Data Analyses; Data Science; Data Set; Development; Education; Education Projects; Educational Curriculum; Educational process of instructing; Engineering; Evaluation; Exercise; Fraud; Generations; Goals; Health; Health Sciences; High School Student; Human Resources; Instruction; Journals; Knowledge; Lead; Learning; Learning Module; Length; Literature; Measures; Mechanics; Methods; Modality; Neurosciences; Neurosciences Research; Outcome; Outcome Measure; Outcomes Research; Paper; Philosophy; Process; Production; Program Accessibility; Publication Bias; Publications; Publishing; Reader; Reading; Reporting; Reproducibility; Research; Research Misconduct; Research Personnel; Risk; Role; Route; Running; Sampling Biases; Science; Scientist; Self-Examination; Series; Shapes; Specific qualifier value; Speed; Statistical Data Interpretation; Statistical Methods; Students; Surveys; Techniques; Technology; Testing; Thinking; Training; United States National Aeronautics and Space Administration; Universities; Washington; Work; adjudication; biomedical scientist; design; experience; guided inquiry; improved; instructor; literacy; next generation; outreach; response; skills; statistics; student training; tool

Project Summary / Abstract As scientific practice evolves in response to exponential increases in data volume, availability of rapid computational statistics, and the so-called “reproducibility crisis”, researchers are developing new methods for collecting and analyze data in rigorous and responsible fashion. The aim of this proposal is to develop three training units for researchers in the neurosciences that will bring learners up to speed on these developments, improving the rigor and quality of their scientific research by deepening their understanding of the role of statistics in biomedical research. Each unit, developed iteratively in a cycle of testing, evaluation, and revision will be designed for online or classroom use suitable for diverse learning styles. Units will comprise a series of short video segments and interactive exercises that lead learners in a process of guided discovery and self-reflection as they move toward a set of well-specified learning goals. Our units will teach neuroscientists to avoid common pitfalls in designing and analyzing data. In the first unit, we address a set of easy-to-make mistakes wherein a researcher alters her plans midway through the process of data analysis. The practice of HARKing—hypothesizing after the results are known—involves testing hypotheses that are formulated after viewing research outcomes. Outcome switching occurs when a study yields negative results based on the pre- specified outcome measures, but other measures are reported instead. The Garden of Forking Paths refers to the latitude that researchers have in shaping a statistical analysis as they go along. The second unit addresses the problem of publication bias, which arises when authors and journals prefer to publish positive results in favor of negative one, and can lead researchers to reduplicate efforts or draw mistaken inferences from published data. The aim of this unit is to make students aware of problem, teach them how to adjust when reading the literature, and suggest strategies for avoiding publication bias in their own work. The third unit will train students how to figure out whether when a statistical analysis rigorous and reliable. Students will learn how to ask “Are the data appropriate what we want to learn?” “Is the choice of statistical test reasonable?” “Are the inferences supported by the evidence?” By developing this set of units, to be included in a broader neuroscience curriculum, we can train a new generation of biomedical scientists who are well-equipped to work with the vast datasets that are becoming available thanks to new research tools and technologies. These scientists will be able to work more accurately, make new discoveries more efficiently, and advance our knowledge in the health and life sciences at a faster rate than ever before.

项目概要/摘要 随着科学实践的发展以应对数据量的指数级增长， 快速计算统计，以及所谓的“再现性危机”，研究人员正在 开发以严格和负责任的方式收集和分析数据的新方法。 该提案的目的是为神经科学研究人员开发三个培训单元 这将使学习者及时了解这些发展，提高学习的严谨性和质量 通过加深对统计在生物医学中的作用的理解来进行科学研究 每个单元都将在测试、评估和修订的周期中迭代开发。 专为在线或课堂使用而设计，适合不同的学习风格。 一系列的短视频片段和互动练习，引导学习者 当他们朝着一系列明确的学习目标迈进时，引导发现和自我反思。 我们的单位将教导神经科学家避免设计和分析数据时的常见陷阱。 第一个单元，我们解决了一系列容易犯的错误，研究人员改变了她的计划 数据分析过程中的HARKing实践——事后假设。 结果是已知的——涉及测试在查看研究后提出的假设 当一项研究根据预期得出负面结果时，就会发生结果转换。 指定了结果衡量标准，但报告了其他衡量标准。 路径是指研究人员在进行统计分析时所拥有的自由度 第二单元解决了作者发表偏见时出现的问题。 期刊更愿意发表积极的结果而不是消极的结果，并且可以引导研究人员 重复工作或从已发布的数据中得出错误的推论 本单元的目的是： 让学生意识到问题，教会他们在阅读文献时如何调整，以及 提出避免自己工作中发表偏见的策略。第三个单元将进行培训。 学生如何判断统计分析时是否严谨可靠。 将学习如何询问“数据是否适合我们想要学习的内容？” 统计检验合理吗？”“推论有证据支持吗？” 通过开发这套单元，将其纳入更广泛的神经科学课程中，我们可以 培养新一代生物医学科学家，他们有能力与广大的科学家合作 由于新的研究工具和技术，数据集变得可用。 科学家将能够更准确地工作，更有效地做出新发现，并且 我们在健康方面的知识和生命科学的发展速度比以往任何时候都快。