Two public symposia on the many faces of reproducibility with invited speakers from outside the university will be held over the course of the project.
Fall 2021 Symposium - December 3, 2021 9am - 6pm
Data and Reproducibility in Light of Diversity, Equity, and Inclusion
This symposium will focus on discussion and details surrounding diversity, equity, and inclusion (DEI) in data to explore the many dimensions of reproducibility in scientific research
More details to come
Frontiers of Reproducibility
October 19, 2020 4 pm - 6:30 pm. This event was held online. A video recording of the exploration of key conceptual issues at the forefront of discussions about reproducibility in the sciences from three distinguished scholars working on these topics is available below.
Replicability and Beyond: Reimagining Science as Truly Open
Alison Ledgerwood, University of California Davis
In recent years, a reform movement has emerged across multiple scientific disciplines seeking to improve the quality of our scientific methods and practices, with a focus on enhancing replicability and transparency. I will discuss how efforts to advance replicability and transparency align with—and in fact, cannot succeed in the absence of—efforts to advance generalizability and inclusiveness in science. I will describe how we can change incentive structures to achieve these goals and how the pandemic affords a unique opportunity to do so.
Should Science be 100% Replicable?
Wendy Wood, University of Southern California
Contrary to much current wisdom, the answer is clearly, “no.” Efforts to make science completely replicable fail to distinguish reproducibility from replicability. Reproducibility refers to getting consistent results from reanalyzing the same data using the same computations and methods of analysis (NASEM, 2019). Yes, complete reproducibility is necessary for reliable scientific knowledge. However, replicability refers to consistent results across studies using different data to test the same scientific question. The replication crisis discourse often overlooks that non-replicability is a normal part of the scientific process and can advance scientific knowledge. Nature is intrinsically varied and complex. Non-replicability can be helpful in identifying limits in current scientific knowledge and technology. I illustrate these claims with failures to replicate that led to the discovery of new phenomena and new insights about variability in established phenomena. The implication for science policy is to set standards that allow for helpful forms of non-replicability.
The Social Limits of Knowledge
James Evans, University of Chicago
The explosive growth of scientists, scientific journals, articles and findings in recent years has exponentially increased the difficulty scientists face in navigating prior knowledge and collectively reasoning over it to drive future advance. This challenge is exacerbated by uncertainty about the reproducibility of published findings. Here I detail a series of investigations using large-scale publication, experimental and clinical data, which reveal how socially, methodologically and institutionally independent research activity is much more likely to replicate than work performed within a singular community of researchers using the same methods and approaches. These findings recommend policies like decentralized collaboration that go against the common practice of channeling biomedical research funding into nonredundant centralized research consortia and institutes rather than dispersing it more broadly. We show how using this pattern can help to decode bias, predict and improve the replicability of published findings. These findings mesh with other work that demonstrates how densely connected research communities are also associated with a reduction in the rate of discoveries and the likelihood of disruptive advance. Together, these findings highlight the limits of knowledge that form within insulated bubbles of scientific discourse.