Developing user-centric training in rigorous research: post-selection inference, publication bias, and critical evaluation of statistical claims.

在严谨的研究中开展以用户为中心的培训：选择后推断、发表偏见和统计声明的批判性评估。

• 批准号: 10721491
• 负责人: Carl T Bergstrom
• 金额: $ 9.39万
• 依托单位: UNIVERSITY OF WASHINGTON
• 依托单位国家: 美国
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-08-01 至 2026-07-31
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/10721491
• 关键词: Accounting; Address; Area; Attention; Awareness; Biological Process; Biological Sciences; Biomedical Research; Books; Collaborations; Coupled; Data; Data Analyses; Data Science; Data Set; Development; Education; Education Projects; Educational Curriculum; Educational process of instructing; Engineering; Evaluation; Exercise; Fraud; Generations; Goals; Health; Health Sciences; High School Student; Human Resources; Instruction; Journals; Knowledge; Lead; Learning; Learning Module; Length; Literature; Measures; Mechanics; Methods; Modality; Neurosciences; Neurosciences Research; Outcome; Outcome Measure; Outcomes Research; Paper; Philosophy; Process; Production; Program Accessibility; Publication Bias; Publications; Publishing; Reader; Reading; Reporting; Reproducibility; Research; Research Misconduct; Research Personnel; Risk; Role; Route; Running; Sampling Biases; Science; Scientist; Self-Examination; Series; Shapes; Specific qualifier value; Speed; Statistical Data Interpretation; Statistical Methods; Students; Surveys; Techniques; Technology; Testing; Thinking; Training; United States National Aeronautics and Space Administration; Universities; Washington; Work; adjudication; biomedical scientist; design; experience; guided inquiry; improved; instructor; literacy; next generation; outreach; response; skills; statistics; student training; tool

Project Summary / Abstract As scientific practice evolves in response to exponential increases in data volume, availability of rapid computational statistics, and the so-called “reproducibility crisis”, researchers are developing new methods for collecting and analyze data in rigorous and responsible fashion. The aim of this proposal is to develop three training units for researchers in the neurosciences that will bring learners up to speed on these developments, improving the rigor and quality of their scientific research by deepening their understanding of the role of statistics in biomedical research. Each unit, developed iteratively in a cycle of testing, evaluation, and revision will be designed for online or classroom use suitable for diverse learning styles. Units will comprise a series of short video segments and interactive exercises that lead learners in a process of guided discovery and self-reflection as they move toward a set of well-specified learning goals. Our units will teach neuroscientists to avoid common pitfalls in designing and analyzing data. In the first unit, we address a set of easy-to-make mistakes wherein a researcher alters her plans midway through the process of data analysis. The practice of HARKing—hypothesizing after the results are known—involves testing hypotheses that are formulated after viewing research outcomes. Outcome switching occurs when a study yields negative results based on the pre- specified outcome measures, but other measures are reported instead. The Garden of Forking Paths refers to the latitude that researchers have in shaping a statistical analysis as they go along. The second unit addresses the problem of publication bias, which arises when authors and journals prefer to publish positive results in favor of negative one, and can lead researchers to reduplicate efforts or draw mistaken inferences from published data. The aim of this unit is to make students aware of problem, teach them how to adjust when reading the literature, and suggest strategies for avoiding publication bias in their own work. The third unit will train students how to figure out whether when a statistical analysis rigorous and reliable. Students will learn how to ask “Are the data appropriate what we want to learn?” “Is the choice of statistical test reasonable?” “Are the inferences supported by the evidence?” By developing this set of units, to be included in a broader neuroscience curriculum, we can train a new generation of biomedical scientists who are well-equipped to work with the vast datasets that are becoming available thanks to new research tools and technologies. These scientists will be able to work more accurately, make new discoveries more efficiently, and advance our knowledge in the health and life sciences at a faster rate than ever before.

项目摘要 /摘要 随着科学实践响应数据量的指数增加，可用性 快速计算统计以及所谓的“可重复性危机”，研究人员是 开发以严格和负责任的方式收集和分析数据的新方法。 该提案的目的是为神经科学研究人员开发三个培训单位 这将使学习者能够加快这些发展的速度，从而提高 他们的科学研究是通过加深对统计在生物医学中的作用的理解 研究。每个单元在测试，评估和修订的周期中迭代开发 专为在线或教室而设计，适用于潜水员学习风格。单位将完成 一系列简短的视频片段和互动练习，使学习者在一个过程中 指导发现和自我反思朝着一系列明确的学习目标。 我们的单位将教神经科学家避免在设计和分析数据时避免常见的陷阱。在 第一个单元，我们解决了一系列易于犯罪的错误，其中研究人员改变了她的计划 中途数据分析过程。习惯 - 在 结果是已知的 - 观看研究后提出的涉及测试假设 结果。当一项研究基于预先的结果产生负面结果时，就会发生结果切换 指定的结果指标，但报告了其他措施。分叉的花园 路径是指研究人员在进行统计分析时所具有的纬度 沿着。第二个单元解决了出版偏差的问题，这是作者时出现的 期刊更喜欢发表积极的结果，而不是负面的结果，并且可以领导研究人员 重新发挥努力或从已发布的数据中提出错误。该单元的目的是 让学生意识到问题，教他们阅读文学时如何调整以及 建议避免在自己的工作中避免出版偏见的策略。第三个单位将训练 学生如何弄清楚何时严格且可靠地进行统计分析。学生 会学习如何问“数据合适我们想学习的内容是否合适？” “是选择 统计测试合理吗？”“证据支持的推论吗？” 通过开发这组单元，将包括在更广泛的神经科学课程中，我们可以 培训一个能够与疫苗合作的新一代生物医学科学家 借助新的研究工具和技术，可以使用的数据集。这些 科学家将能够更准确地工作，使新发现更有效，并且 比以往任何时候都更快地促进我们在健康和生活科学方面的知识。