Reproducibility: Gold or Fool's Gold in Digital Social Research?

This is a Keynote talk from the Oxford Internet Institute's Symposium "Social Science and Digital Research: Interdisciplinary Insights", held in Oxford on 12 March 2012. Data sharing has become a core tenet of science policy in the UK, the US, and elsewhere. Among the rationales for sharing data is improving the ability to reproduce or to replicate research. Reproducibility is an oft-stated "gold standard" for science, yet it is a problematic rationale for sharing research data. Sociologists of science have described the difficulties of verifying, let alone reproducing, scientific results, since the 1970s. While most sciences are experiencing a data deluge, the characteristics and practices associated with data vary widely, with different requirements for replication. Reproducibility concerns underlie peer review, identification of fraud, bio-security, and publication practices. The role of data in reproducing science lies at the intersection of e-Science, practice, and policy, and thus is a significant problem to be addressed by digital social research.