Sackler Colloquia banner

Reproducibility of Research: Issues and Proposed Remedies

March 8-10, 2017; Washington, D.C.
Organized by David B. Allison, Richard Shiffrin, and Victoria Stodden


This colloquium was held in Washington, D.C. on March 8-10, 2017.

This colloquium brought together scientists and researchers from multiple disciplines to lay out the scope of the problem of reproducibility in a more tactical way that permits each problematic aspect to be measured, assessed for baseline levels, targeted with proposed interventions to reduce the occurrence, and monitored for improvement.

Videos of the talks are available on the Sackler YouTube Channel. Some speakers have delayed making their talks public due to publication embargos.


March 8, 2017
Colloquium Call to Order, David Allison, University of Alabama at Birmingham
Welcoming Remarks, Marcia McNutt, President, National Academy of Sciences
Remembering Stephen Fienberg, Robert Groves, Provost, Georgetown University
Framing the Issues, Victoria Stodden, University of Illinois 


Issues of Design & Measurement, Kay Dickersin, Johns Hopkins University

Issues of Analysis, David B. Allison, University of Alabama at Birmingham

Taking Stock/Making Change: Surveying Practices, Assessing Perspectives, and Re-training Personnel as methods of supporting reproducibility in a learning health system, Madhu Mazumdar, Icahn School of Medicine at Mount Sinai Medical College

Incomplete Reporting in Primary Scientific Literature, Lehana Thabane, McMaster University

Honest Learning for the Healthcare System: Large-Scale Evidence from Real-World Data, David Madigan, Columbia University

Distortion of Research Results in Primary Scientific Literature and Beyond, Isabelle Boutron, Centre d'Epidémiologie Clinique, Hôpital Hôtel Dieu

PANEL DISCUSSION – Moderator: Andrew W. Brown, UAB
Panelists: Inder Verma (PNAS), Phil Campbell (Nature), Kelvin Droegemeier (Oklahoma University), Veronique Kiermer (PloS), Ginny Barbour (COPE)
Annual Sackler Lecture: The Challenge of Reproducibility in the Biomedical Sciences, Randy Schekman, University of California, Berkeley.

March 9, 2017—REMEDIES

Raw Data Depositing and Sharing, Victoria Stodden, University of Illinois

The Role of Registries and Repositories, Roberta W. Scherer, Johns Hopkins University

Statistical Power and Evidence in the Psychological Literature, Joachim Vandekerckhove, University of California, Irvine

Rigorous Assessment of Whether A Result can be Said to Have Been Replicated, Yoav Benjamini, Tel Aviv University

Training the Current and the Next Generations in Statistics, Emery Brown, Massachusetts Institute of Technology

Biomedical replication studies, Elizabeth Iorns, Science Exchange

Agents of Cultural Behavior Change, Susan T. Fiske, Princeton University

Post-Publication Peer-Review and Certificate Systems, Hilda Bastian, National Center for Biotechnology Information, NIH

Science Journalism and Public Dialog, Trevor Butterworth, Sense About Science USA

Government Perspectives, Catherine Woteki, Former Undersecretary for Research, Education and Economics, USDA

PANEL DISCUSSION – Moderator: Kathryn A. Kaiser, University of Alabama at Birmingham
Panelists: Daniele Fanelli, Stanford University; Jeffrey Flier, Harvard University

March 10, 2017—RESEARCH GOALS

The Problem with Remedies: The Need to Study Unintended Consequences, Richard M. Shiffrin, Indiana University, Bloomington

Minimizing Inadvertent Distortion of Science by Scientists and Media, Kathleen Hall Jamieson, University of Pennsylvania

Systems Matter: Research Environments and Institutional Integrity, C. K. Gunsalus, University of Illinois

Testing Interventions at the Levels of the Journal, David Moher, University of Ottawa

Funder’s Perspective, Richard Nakamura, NIH Center for Scientific Review

Improving Reproducibility, Brian Nosek, University of Virginia, Center for Open Science (COS)

New statistical approaches to reproducibility, Giovanni Parmigiani, Dana Farber Cancer Institute, Harvard University

Sharing scientific data and replicability, Keith Baggerly, MD Anderson Cancer Center

Closing Remarks, Richard Shiffrin, Indiana University, Bloomington

Powered by Blackbaud
nonprofit software