Nothing new! This too has been studied and reported frequently over the past several decades!
"A sweeping project involving hundreds of researchers in several dozen countries showed that across the social sciences, the findings of roughly half of all papers cannot be replicated independently, and there’s no reliable way to tell in advance which ones will falter. Called Systematizing Confidence in Open Research and Evidence (SCORE), the effort investigated more than 100 papers published in dozens of leading journals in business, economics, education, political science, psychology, and sociology. The replication success rate—49% for the 164 papers evaluated, reported today in Nature—is consistent with findings from previous studies in individual fields such as psychology, suggesting the problem is pervasive in the social sciences. ..."
From the abstract:
"Pursuing replicability — independent evidence for previous claims — is important for creating generalizable knowledge. Here we attempted replications of 274 claims of positive results from 164 quantitative papers published from 2009 to 2018 in 54 journals in the social and behavioural sciences. Replications were high powered on average to detect the original effect size (median of 99.6%), used original materials when relevant and available, and were peer reviewed in advance through a standardized internal protocol.
Replications showed statistically significant results in the original pattern for 151 of 274 claims (55.1% (95% confidence interval (CI) 49.2–60.9%)) and for 80.8 of 164 papers (49.3% (95% CI 43.8–54.7%)), weighed for replicating multiple claims per paper.
We observed modest variation in replication rates across disciplines (42.5–63.1%), although some estimates had high uncertainty. The median Pearson’s r effect size was 0.25 (95% CI 0.21–0.27) for original studies and 0.10 (95% CI 0.09–0.13) for replication studies, an 82.4% (95% CI 67.8–88.2%) reduction in shared variance. Thirteen methods for evaluating replication success provided estimates ranging from 28.6% to 74.8% (median of 49.3%). Some decline in effect size and significance is expected based on power to detect original effects and regression to the mean because we replicated only positive results.
We observe that challenges for replicability extend across social–behavioural sciences, illustrating the importance of identifying conditions that promote or inhibit replicability."
Investigating the replicability of the social and behavioural sciences (no public access)
No comments:
Post a Comment