Tuesday, August 11, 2020

Researchers find 'inconsistent' benchmarking across 3,867 AI research papers

A case of very sloppy, sensationalist journalism!



Did the journalist not read what the affiliations are of the authors? "Institute for Artificial Intelligence and Decision Support, Center for Medical Statistics,Informatics, and Intelligent Systems, Medical University of Vienna, Vienna, Austria"

Obviously, these authors are not experts in AI or machine learning!



The authors sourced their sample papers from "Paper with Code"

Did the journalist not know that "Anyone can contribute!" (see About)



At the end of the article we finally learn: "The researchers concede their decision to analyze preprints as opposed to papers accepted to scientific journals could skew the results of their study."

That is exactly the case!



Researchers find 'inconsistent' benchmarking across 3,867 AI research papers | VentureBeat



Here is the underlying preprint arXiv paper: https://arxiv.org/abs/2008.02577

No comments: