A new massively large text training corpus for language models

Good news! All the literature ever written at your fingertips! The modern Library of Alexandria (one of the ancient wonders of the world)!

"... Harvard unveiled the Harvard Library Public Domain Corpus, nearly 1 million copyright-free books that were digitized as part of the Google Books project. That’s five times as many volumes as Books3, which was used to train large language models including Meta’s Llama 1 and Llama 2 but is no longer available through lawful channels. ...

For now, it’s available only to current Harvard students, faculty, and staff. The university is working with Google to distribute it widely.  ..."

When Good Models Do Bad Things, What Users Really Want, More Training Data!, Better Model Merging

Harvard Library Public Domain Corpus "Harvard Library offers the Harvard community free access to the Harvard Library Public Domain Corpus, a collection of approximately one million digitized public domain books. "