An empirical assessment of transparency and reproducibility-related research practices in the social sciences (2014-2017)

Abstract

Serious concerns about research quality have catalyzed a number of reform initiatives intended to improve transparency and reproducibility and thus facilitate self-correction, increase efficiency, and enhance research credibility. Meta-research has evaluated the merits of individual initiatives; however, this may not capture broader trends reflecting the cumulative contribution of these efforts. In this study, we evaluated a broad range of indicators related to transparency and reproducibility in a random sample of 198 articles published in the social sciences between 2014 and 2017. Few articles indicated availability of materials (1596, 16% [95% confidence interval, 9% to 23%]), protocols (0/103), raw data (8103, 8% [2% to 15%]), or analysis scripts (3103, 3% [1% to 6%]), and no studies were pre-registered (0/103). Some articles explicitly disclosed funding sources (or lack of; 72179, 40% [33% to 48%]) and some declared no conflicts of interest (32179, 18% [13% to 24%]). Replication studies were rare (2103, 2% [0% to 4%]). Few studies were included in evidence synthesis via systematic review (696, 6% [3% to 11%]) or meta-analysis (296, 2% [0% to 4%]). Slightly less than half the articles were publicly available (95198, 48% [41% to 55%]). Minimal adoption of transparency and reproducibility-related research practices could be undermining the credibility and efficiency of social science research. The present study establishes a baseline that can be revisited in the future to assess progress.

Publication
MetaArXiv
Date