OpenAlex, a new competitor for Semantic Scholar and Google Scholar

Good news! The website will be launched in February 2022! I am  looking forward to using it like Semantic Scholar and Google Scholar.

This web search engine also provides a public API so I can automate my queries! 

It appears this will be some kind of a successor of Microsoft Academic, which unfortunately shut down at the end of 2021

"... Our two most important data sources are MAG [Microsoft Academic Graph] and Crossref. ..."

Welcome! - OpenAlex documentation

Bye, Bye Microsoft Academic

As of today (1/1/2022), the great website Microsoft Academic (http://academic.microsoft.com/) is gone!

I was hoping to the last minute that someone would rescue this great site from being discontinued. Unfortunately, a knight in shining armor did not show up this time.

Microsoft Academic had several serious advantages over its competitors Google Scholar and Semantic Scholar. I have intensely worked will all three websites over the past two years or so almost on a daily basis to study artificial intelligence and machine learning.

For example, MA would clearly delineate in which journals and what year an article was published. This was very helpful if an article was published in multiple venues at different times. The competitors fail to do that.

The other major competitor, Prophy, a European Union product, is rather bureaucratic and cumbersome. In short, no match!

Microsoft Academic announced its shut down by end of 2021

This is really bad news! Microsoft Academic has some strength and is superior  in some respects to its competitors like Google Scholar or Semantic Scholar. I have used it very frequently during the past two years  for my research of artificial intelligence, computer vision, machine learning, computational linguistics etc. 

Microsoft Academic excels compared to its competitors e.g. in:

1. It is much easier to learn if a research article was published in multiple venues, which venues and when it was published in that venue
2. The information provided about publications is usually of high quality. Compared to Semantic Scholar it has far fewer quality issues. 

One can only hope that Google Scholar will seize the opportunity. Unfortunately, Google Scholar has for several years refused automatic requests to retrieve information and it does not provide an API to do so! Very regrettable! I know, because my efforts to write a Python program to retrieve information has been blocked by Google Scholar!

"Microsoft Academic Website: No longer accessible after Dec. 31, 2021"

Next Steps for Microsoft Academic – Expanding into New Horizons (What an euphemistic announcement!)

Google Scholar, Microsoft Academic, and Semantic Scholar Annoyances

Since I use all three services, I will try to compare their strengths and weaknesses etc. I will use following abbreviations GS (Google Scholar), MA (Microsoft Academic), and S2 (Semantic Scholar). More items will be added in the future.

1. GS, MA, and S2: sometimes revise down the citations count for seminal papers by a few thousand or so within several months. E.g. noticed it today (4/12/2021) for "2014 Very Deep Convolutional Networks for Large-Scale Image Recognition by Karen Simonyan, Andrew Zisserman" it changed from a maximum of over 61,000 to now 57,000
2. GS: does not provide a unique identifier for the articles (4/11/2021)
3. MA and S2: The count of references per paper often deviate dramatically. GS does not publish a references count. This certainly contributes to messed up citations counts for papers!
   Example: Generative Adversarial Nets, a highly cited seminal paper: MA 33 reference count; S2 41 (2/25/2021)
4. GS, MA, and S2: Their count of papers per author often deviate dramatically. Google Scholar does not even publish the number in a straight forward  manner. Example: Joshua Bengio: GS 1075 total publications; MA 973; S2 (S2 has three author pages for Yoshua Bengio) 838 (2/25/2021)
5. GS: Lacks profiles for highly cited authors e.g. Emanuel Parzen (1919-2016). Google Search also does not link to Google Scholar profiles. MA and S2 do link to author profiles much more consistently (6/20/2021)
6. S2: Unfortunately, S2 contains hundreds if not thousands of junk entries! I provide feedback on them as I find them. However, there are way too many junk entries. Junk entries have consequences: E.g. they often screw up citation counts of the paper itself and the references list of other papers (6/20/2021)
7. GS, MA, and S2: All three have significant problems to distinguish book chapters from single research articles. GS search sometimes only returns a "[Citation]" as a research result (sometimes with foreign language characters such as Japanese, very odd)! Consequences: It leads sometimes to duplicate entries and probably also to messed up reference list for other papers (6/20/2021)
8. GS: Does not provide a list of the paper references (6/24/2021)
9. MA & S2: Only these two allow to sort the references of a paper by several criteria. However, you cannot sort by original order or by last name of first author (6/24/2021)
10. GS: I noticed now several times that the Follow this Author function for new articles by author does not work reliably. It misses new papers. I speculate, but it may have something to do whether the author is the first or last author or somewhere in between. This is very annoying! (8/9/2021)

Microsoft Academic was down today

I tried dozens of searches for exact title match today with a few hours and they all failed. Intermittently, it seemed to work for one or two searches before it failed again. Was this by any chance related to Microsoft's trouble with Exchange or with hacker attacks?

MA failed to inform its users. Here is a screen print of a typical search failure:

Microsoft Academic Annoyances

Posted: 2/19/2020  Updated: 

I have been using Microsoft Academics for about 1.5 years or longer.

List of annoyances:

1. A user’s Reading List has a ridiculous maximum limit of 1000 papers. It makes no sense in this day and age!
2. I have lately noticed frequent stability issues with the website after entering paper titles in the search bar (as of 2/20/2020). Microsoft Academic fixed this issue by early March 2020, they even provided some feedback to me about the issue after me contacting them and that it was being remediated
   
3. This is lousy done! Would be a very useful tool to drill down by topic [e.g. Active Learning] and then select certain journals! (3/17/2020)
   You cannot read the names of the top journals!
   You cannot select multiple top journals at once!
   
4. 
   

Notes:

1. The list of annoyances will be amended as time goes by

Google Mechanical Turk

Posted: 5/11/2019 Updated: 8/2/2019, 6/10/2019/6/7/2019, 6/6/2019

Update Of 8/2/2019

I don’t think I mentioned it before, but I am a paying customer of Google Drive for about 2 years or longer and I log into Google Scholar with my Google account. Despite that Google has the freaking audacity to ask me whether I am a robot. This utter nonsense is literally pissing me off!

Meanwhile, I discovered that Microsoft Academic is not such a bad substitute for Google Scholar. It also has its quirks, but also its strength, e.g. it splits citation counts by publication. Long live competition!

Update Of 6/10/2019

Image classification tasks (Google Mechanical Turk without pay) now also effective when using incognito window in Google Chrome Browser. EXTREMELY ANNOYING!

Update Of 6/7/2019

I may have found a workaround this issue: When using the incognito window in Google Chrome browser, you are apparently not prompted to participate in the Google Mechanical Turk assignments!

Hooray!

Update Of 6/6/2019

Google Scholar is now inundating me every day several times with their image classification tasks!

And if you refuse to play along nicely, Google will not budge!

EXTREMELY ANNOYING!

Original Post

I assume many readers are familiar with the Amazon Mechanical Turk. According to Amazon: “.... a crowdsourcing marketplace that makes it easier for individuals and businesses to outsource their processes and jobs ”. At least you get paid, if little for your work.

Google has, to my knowledge, recently introduced what I call the Google Mechanical Turk. Google Scholar will subject you to image classification tasks if you query Google Scholar frequently for scholars or research papers. Google claims it is a security measure to foil bots. Unfortunately, you do not get paid for this work at all! Shame on Google!

P.S.

As a teenager I saw the Mechanical Turk (a.k.a. Chess Automaton) in a encyclopedia of historical technology. For its time, it was a very impressive achievement, but it was a magician’s trick. 

More famous (less magic and more mechanical engineering) were perhaps the Jaquet-Droz’s Three Automatons (see e.g. here.