The titles of at least 3200 research papers directly quote or make puns related to The Beatles song names or lyrics

Amazing stuff! One of the greatest pop music bands of all times lives on! Or how contagious is the Beatlemania?

Tells you something about the age of the researchers! Just kidding!

Apparently, the authors of medical articles make the most use of the song lyrics/titles of the Beatles (see chart below)

However, it is a drip in the bucket. "The worldwide annual output of science and engineering research articles totals roughly 3.3 million peer-reviewed papers per year" (Google Search)

"Scientists seeking to make an academic paper more engaging often get by with a little help from The Beatles . References to the “Fab Four” are here, there and everywhere, researchers have found: The titles of at least 3200 papers directly quote or make puns related to their song names or lyrics ..."

From the abstract:

"The Beatles’ cultural influence reaches well beyond music into television, fashion, and global media.

As we show, it reaches into scientific publishing too. This study documents how often Beatles song titles and lyrics appear in academic article titles, identifying 2,237 exact references and 963 instances of creative wordplay in data from Scopus.

Working from a curated list of Beatles songs and lyrics, we combine automated title queries with large-language-model screening, followed by manual verification, to detect both exact references and wordplay.

To our knowledge, this is the first study to systematically identify wordplay on Beatles song titles and lyrics in scientific article titles. We report which titles and lyrics are referenced most often, trace how their use has grown over the decades, and show that this growth has outpaced the expansion of the scientific literature as a whole while varying systematically across disciplines.

The result is the first large-scale, cross-disciplinary account of how the Beatles have left their mark on scientific writing."

ScienceAdviser

How science gets by with a little help from the Beatles: Detecting cultural wordplay in scientific literature using large language models (open access)

Fig 1. Comparison of exact matches of Beatles song titles in academic article titles on a log-log scale: Scopus vs. Google Scholar.

Fig 3. Subject areas of articles referencing the 10 most frequently matched Beatles song titles.

Notes on The Bitter Lesson of Diffusion Language Models for Agentic Workflows: A Comprehensive Reality Check

Food for thought! Just finished reading this paper. A critical paper by Dacheng Tao and his team!

"... From the above analysis, we observe a fundamental limitation of dLLMs [Diffusion-based Large Language Models]: despite their efficiency gains, parallel decoding weakens causal dependency and induces fuzzy intermediate states, hindering stable commitment to partial plans or structured outputs. ...

As a result, dLLMs perform poorly on long-horizon reasoning and strictly structured tasks ..."

From the abstract:

"The pursuit of real-time agentic interaction has driven interest in Diffusion-based Large Language Models (dLLMs) as alternatives to auto-regressive backbones, promising to break the sequential latency bottleneck.

However, does such efficiency gains translate into effective agentic behavior? In this work, we present a comprehensive evaluation of dLLMs (e.g., LLaDA, Dream) across two distinct agentic paradigms:

Embodied Agents (requiring long-horizon planning) and 

Tool-Calling Agents (requiring precise formatting).

Contrary to the efficiency hype, our results on Agentboard and BFCL reveal a "bitter lesson": current dLLMs fail to serve as reliable agentic backbones, frequently leading to systematically failure.

(1) In Embodied settings, dLLMs suffer repeated attempts, failing to branch under temporal feedback.

(2) In Tool-Calling settings, dLLMs fail to maintain symbolic precision (e.g. strict JSON schemas) under diffusion noise.

To assess the potential of dLLMs in agentic workflows, we introduce DiffuAgent, a multi-agent evaluation framework that integrates dLLMs as plug-and-play cognitive cores.

Our analysis shows that dLLMs are effective in non-causal roles (e.g., memory summarization and tool selection) but require the incorporation of causal, precise, and logically grounded reasoning mechanisms into the denoising process to be viable for agentic tasks."

[2601.12979] The Bitter Lesson of Diffusion Language Models for Agentic Workflows: A Comprehensive Reality Check

On Temporal structure of natural language processing in the human brain corresponds to layered hierarchy of large language models

Should it not be the other way around?

The age of machine learning & AI has just begun, but some researchers in this field are already downgrading or subsuming the human brain.

From the abstract:

"Large Language Models (LLMs) offer a framework for understanding language processing in the human brain. Unlike traditional models, LLMs represent words and context through layered numerical embeddings.

Here, we demonstrate that LLMs’ layer hierarchy aligns with the temporal dynamics of language comprehension in the brain. 

Using electrocorticography (ECoG) data from participants listening to a 30-minute narrative, we show that deeper LLM layers correspond to later brain activity, particularly in Broca’s area and other language-related regions. We extract contextual embeddings from GPT-2 XL and Llama-2 and use linear models to predict neural responses across time.

Our results reveal a strong correlation between model depth and the brain’s temporal receptive window during comprehension.

We also compare LLM-based predictions with symbolic approaches, highlighting the advantages of deep learning models in capturing brain dynamics. We release our aligned neural and linguistic dataset as a public benchmark to test competing theories of language processing."

Temporal structure of natural language processing in the human brain corresponds to layered hierarchy of large language models | Nature Communications (open access)

Fig. 2: Temporal dynamics of layer-wise encoding for correctly predicted words in IFG.

Enduring patterns in world's languages: One-third of grammatical 'universals' stand up to rigorous testing

Amazing stuff!

"... An international team ... used Grambank, the world's most comprehensive database of grammatical features, to test 191 proposed universals across more than 1,700 languages. Traditionally, linguists have attempted to circumvent the genealogical and geographic non-independence of languages by sampling widely separated languages.

However, sampling can fail to remove all dependencies, reduce statistical power and does not identify historical pathways. The Bayesian spatio-phylogenetic analyses used by the authors accounted for both the genealogical and geographic non-independence of languages—a level of statistical rigor rarely achieved in previous work. ...

The study found strong evidence for patterns involving word order (such as whether verbs precede or follow objects) and hierarchical universals (such as dependencies in which arguments are marked in grammatical agreement). The patterns predicted by the supported universals have evolved repeatedly across the world's languages, suggesting deep-rooted constraints in how humans structure communication. ..."

"To the Point

• Linguistic universals: Of the 191 proposed linguistic universals, about one-third are statistically supported across more than 1,700 languages.
• A wealth of data and state-of-the-art statistical methods: Using Grambank and Bayesian statistical models that control for genealogical and geographic influences, the strongest evidence emerges for patterns of word order and hierarchical agreement.
• Evolutionary framework: The repeated evolution of these patterns suggests there are shared cognitive and communicative constraints, which narrows the search for truly universal features of human language.

..."

From the abstract:

"Human languages show astonishing variety, yet their diversity is constrained by recurring patterns. Linguists have long argued over the extent and causes of these grammatical ‘universals’.

Using Grambank—a comprehensive database of grammatical features across the world’s languages—we tested 191 proposed universals with Bayesian analyses that account for both genealogical descent and geographical proximity.

We find statistical support for about a third of the proposed linguistic universals.

The majority of these concern word order and hierarchical universals: two types that have featured prominently in earlier work.

Evolutionary analyses show that languages tend to change in ways that converge on these preferred patterns.

This suggests that, despite the vast design space of possible grammars, languages do not evolve entirely at random.

Shared cognitive and communicative pressures repeatedly push languages towards similar solutions."

Enduring patterns in world's languages: One-third of grammatical 'universals' stand up to rigorous testing

Enduring patterns in the world’s languages (original news release) "New study finds one-third of grammatical ‘universals’ stand up to rigorous testing"

Enduring constraints on grammar revealed by Bayesian spatiophylogenetic analyses (open access)

The evolution of a word-order universal on the global language tree. In our analysis of the universal “With overwhelmingly greater than chance frequency, languages with normal subject–object–verb order are postpositional”, the absence or presence of the two features defines the ‘state’: state 11 (red) is the prediction made by the universal; in state 00 (black), both features are absent; in states 01 (orange) and 10 (light blue), one feature is absent and the other is present. The ancestral state reconstruction shows that in multiple language families and areas, pathways of language change repeatedly lead to the predicted outcome. 

Fig. 2: Median natural log BF and their 95% HDI from the BayesTraits analyses showing support for co-evolutionary models.

On Tracing Multilingual Representations in LLMs with Cross-Layer Transcoders

This could be an interesting paper by Bernhard Schölkopf and his team.

From the abstract:

"Multilingual Large Language Models (LLMs) can process many languages, yet how they internally represent this diversity remains unclear. Do they form shared multilingual representations with language-specific decoding, and if so, why does performance still favor the dominant training language? 

To address this, we train a series of LLMs on different mixtures of multilingual data and analyze their internal mechanisms using cross-layer transcoders (CLT) and attribution graphs.

Our results provide strong evidence for pivot language representations: the model employs nearly identical representations across languages, while language-specific decoding emerges in later layers.

Attribution analyses reveal that decoding relies in part on a small set of high-frequency language features in the final layers, which linearly read out language identity from the first layers in the model.

By intervening on these features, we can suppress one language and substitute another in the model's outputs.

Finally, we study how the dominant training language influences these mechanisms across attribution graphs and decoding pathways. We argue that understanding this pivot-language mechanism is crucial for improving multilingual alignment in LLMs."

[2511.10840] Tracing Multilingual Representations in LLMs with Cross-Layer Transcoders

On ReviewerToo: Should AI Join The Program Committee? A Look At The Future of Peer Review

Let AI do the peer review so humans can spend more time on the beach? Maybe Hugo Larochelle and his team are onto something! They seem to be serious! 😊

Caveat: I did not read this paper.

From the abstract:

"Peer review is the cornerstone of scientific publishing, yet it suffers from inconsistencies, reviewer subjectivity, and scalability challenges. We introduce ReviewerToo, a modular framework for studying and deploying AI-assisted peer review to complement human judgment with systematic and consistent assessments.

ReviewerToo supports systematic experiments with specialized reviewer personas and structured evaluation criteria, and can be partially or fully integrated into real conference workflows.

We validate ReviewerToo on a carefully curated dataset of 1,963 paper submissions from ICLR 2025, where our experiments with the gpt-oss-120b model achieves 81.8% accuracy for the task of categorizing a paper as accept/reject compared to 83.9% for the average human reviewer.

Additionally, ReviewerToo-generated reviews are rated as higher quality than the human average by an LLM judge, though still trailing the strongest expert contributions. Our analysis highlights domains where AI reviewers excel (e.g., fact-checking, literature coverage) and where they struggle (e.g., assessing methodological novelty and theoretical contributions), underscoring the continued need for human expertise.

Based on these findings, we propose guidelines for integrating AI into peer-review pipelines, showing how AI can enhance consistency, coverage, and fairness while leaving complex evaluative judgments to domain experts. Our work provides a foundation for systematic, hybrid peer-review systems that scale with the growth of scientific publishing."

[2510.08867] ReviewerToo: Should AI Join The Program Committee? A Look At The Future of Peer Review

On Do Language Models Use Their Depth Efficiently?

Seems to be an interesting paper by Christopher Manning and Christopher Potts [no mistake, the two last authors have the same first name]!

At least this paper raises some relevant questions about the purpose and value of ever deeper models.

Caveat: I have not yet read this paper.

From the abstract:

"Modern LLMs are increasingly deep, and depth correlates with performance, albeit with diminishing returns. However, do these models use their depth efficiently? Do they compose more features to create higher-order computations that are impossible in shallow models, or do they merely spread the same kinds of computation out over more layers?

To address these questions, we analyze the residual stream of the Llama 3.1 and Qwen 3 family of models. We find: 

First, comparing the output of the sublayers to the residual stream reveals that layers in the second half contribute much less than those in the first half, with a clear phase transition between the two halves.

Second, skipping layers in the second half has a much smaller effect on future computations and output predictions.

Third, for multihop tasks, we are unable to find evidence that models are using increased depth to compose subresults in examples involving many hops.

Fourth, we seek to directly address whether deeper models are using their additional layers to perform new kinds of computation. 

To do this, we train linear maps from the residual stream of a shallow model to a deeper one.

We find that layers with the same relative depth map best to each other, suggesting that the larger model simply spreads the same computations out over its many layers. All this evidence suggests that deeper models are not using their depth to learn new kinds of computation, but only using the greater depth to perform more fine-grained adjustments to the residual. This may help explain why increasing scale leads to diminishing returns for stacked Transformer architectures."

[2505.13898] Do Language Models Use Their Depth Efficiently?

All the hype about AI in one image

I thought, current large language models are multilingual up to hundreds of languages? 😊 What a disappointment!

This was my prompt today!

A new massively large text training corpus for language models

Good news! All the literature ever written at your fingertips! The modern Library of Alexandria (one of the ancient wonders of the world)!

"... Harvard unveiled the Harvard Library Public Domain Corpus, nearly 1 million copyright-free books that were digitized as part of the Google Books project. That’s five times as many volumes as Books3, which was used to train large language models including Meta’s Llama 1 and Llama 2 but is no longer available through lawful channels. ...

For now, it’s available only to current Harvard students, faculty, and staff. The university is working with Google to distribute it widely.  ..."

When Good Models Do Bad Things, What Users Really Want, More Training Data!, Better Model Merging

Harvard Library Public Domain Corpus "Harvard Library offers the Harvard community free access to the Harvard Library Public Domain Corpus, a collection of approximately one million digitized public domain books. "

On The Super Weight in Large Language Models

Very recommendable! Nice paper by mostly Apple researchers. The senior authors of this paper are largely unknown.

This paper clearly demonstrates that large language models are overhyped! Some more serious research to understand how machine learning models work is in order.

From the abstract:

"Recent works have shown a surprising result: a small fraction of Large Language Model (LLM) parameter outliers are disproportionately important to the quality of the model. LLMs contain billions of parameters, so these small fractions, such as 0.01%, translate to hundreds of thousands of parameters. In this work, we present an even more surprising finding: Pruning as few as a single parameter can destroy an LLM's ability to generate text -- increasing perplexity by 3 orders of magnitude and reducing zero-shot accuracy to guessing. We propose a data-free method for identifying such parameters, termed super weights, using a single forward pass through the model. We additionally find that these super weights induce correspondingly rare and large activation outliers, termed super activations. When preserved with high precision, super activations can improve simple round-to-nearest quantization to become competitive with state-of-the-art methods. For weight quantization, we similarly find that by preserving the super weight and clipping other weight outliers, round-to-nearest quantization can scale to much larger block sizes than previously considered. To facilitate further research into super weights, we provide an index of super weight coordinates for common, openly available LLMs."

"... However, not all outliers are equally important. In this paper, we study a tiny yet important set of outliers in LLMs, termed super weights. In Llama-7B, pruning the super weight, a single scalar, completely destroys the model’s ability to generate text; the average accuracy of zero-shot down-stream tasks effectively plummets to zero. Conversely, pruning the other top 7,000 outliers, including outliers that are larger than the super weight, affects no more than a few percentage points. Intriguingly, super weights behave similarly across model families and sizes. For one, the super weight is always found in the mlp.down proj weight, always in an early layer. 

We also find that the super weight amplifies input activation inliers to ultimately produce the exceptionally large magnitude activation observed by Sun et al. (2024) – we term this the super activation. This super activation persists throughout the model at exactly the same magnitude and position regardless of the prompt, and we find this is uniquely enabled by skip connections. Finally, super weights suppress stopword likelihood. Taken together, pruning the super weight destroys quality by dampening the super activation and shifting almost all logit probability mass to stopwords. ..."

[2411.07191] The Super Weight in Large Language Models

On Self-Rewarding Language Models

Recommendable! Self improving by self rewarding, a new approach!

Superhuman feedback, because human feedback is too confining or becomes a bottleneck! 😊

From the abstract:

"We posit that to achieve superhuman agents, future models require superhuman feedback in order to provide an adequate training signal. Current approaches commonly train reward models from human preferences, which may then be bottlenecked by human performance level, and secondly these separate frozen reward models cannot then learn to improve during LLM training. In this work, we study Self-Rewarding Language Models, where the language model itself is used via LLM-as-a-Judge prompting to provide its own rewards during training. We show that during Iterative DPO training that not only does instruction following ability improve, but also the ability to provide high-quality rewards to itself. Fine-tuning Llama 2 70B on three iterations of our approach yields a model that outperforms many existing systems on the AlpacaEval 2.0 leaderboard, including Claude 2, Gemini Pro, and GPT-4 0613. While there is much left still to explore, this work opens the door to the possibility of models that can continually improve in both axes."

[2401.10020] Self-Rewarding Language Models

On OLMoE: Open Mixture-of-Experts Language Models

Very recommendable! A very impressive and comprehensive research paper with over 200 references!

From the abstract:

"We introduce OLMoE, a fully open, state-of-the-art language model leveraging sparse Mixture-of-Experts (MoE). OLMoE-1B-7B has 7 billion (B) parameters but uses only 1B per input token. We pretrain it on 5 trillion tokens and further adapt it to create OLMoE-1B-7B-Instruct. Our models outperform all available models with similar active parameters, even surpassing larger ones like Llama2-13B-Chat and DeepSeekMoE-16B. We present various experiments on MoE training, analyze routing in our model showing high specialization, and open-source all aspects of our work: model weights, training data, code, and logs."

[2409.02060] OLMoE: Open Mixture-of-Experts Language Models

Is It AI? Peer Reviewers Struggle to Distinguish large language models (LLMs) From Human Writing

Signs of our times!

Perhaps essay contests and the like are fast becoming obsolete in the age of AI!

"A team ... by hosting an essay contest for the journal Stroke that included both AI and human submissions. The researchers found that reviewers struggled to accurately distinguish human from AI essays when authorship was blinded.  ..."

Is It AI? Peer Reviewers Struggle to Distinguish LLMs From Human Writing < Yale School of Medicine "Large language models (LLMs) such as ChatGPT have grown so advanced that they can even pass the US Medical Licensing Exam. But how good are peer reviewers at AI detection, and how does the use of AI affect their perceptions of the work?"

Stanford University: Covert Racism in AI: How Language Models Are Reinforcing Outdated Stereotypes. Really!

I am so sick and tired of this often repeated propaganda and demagoguery of biased and racist language models! These are obvious attempts at censorship and taking control of AI by ideologists!

Of course, the output of LLMs is biased. What else do you expect! That is a given! These models are trained on huge public and private corpora like the Web etc. loaded with biases and racist content. Bias and racism is an unfortunate human condition mutually affecting all skin colors!

What does the author mean by "outdated stereotypes"? Is she not herself using some very strange stereotypes like "African American English" (AAE)?, whatever that is. Probably, if you ask any black American about whether he or she speaks AAE he/she would think you are a weirdo! 

The ivory tower of Stanford University and other racism obsessed elite universities! Ivory tower another one of those cliches or stereotypes with African connotation?

Caveat: I did not read the article.

"... But in a new study, Stanford researchers find that these models still surface extreme racist stereotypes dating from the pre-Civil Rights era. ..."

Covert Racism in AI: How Language Models Are Reinforcing Outdated Stereotypes "Despite advancements in AI, new research reveals that large language models continue to perpetuate harmful racial biases, particularly against speakers of African American English."

Credits: Stanford Institute for Human-Centered Artificial Intelligence newsletter

AI generates more novel and exciting research ideas than human experts, a Stanford University study finds

AI has the power to make any human look like they are demented! Get used to it! Just kidding!

Since Albert Einstein's are born very rarely, AI to the rescue! 😊

"... But as with artistic expression, music composition, coding, understanding subtext and body language, and any number of other emergent abilities, today's multimodal AIs do appear to be able to generate novel research – more novel on average than their human counterparts. ..."

From the abstract:

"Recent advancements in large language models (LLMs) have sparked optimism about their potential to accelerate scientific discovery, with a growing number of works proposing research agents that autonomously generate and validate new ideas. Despite this, no evaluations have shown that LLM systems can take the very first step of producing novel, expert-level ideas, let alone perform the entire research process. We address this by establishing an experimental design that evaluates research idea generation while controlling for confounders and performs the first head-to-head comparison between expert NLP researchers and an LLM ideation agent. By recruiting over 100 NLP researchers to write novel ideas and blind reviews of both LLM and human ideas, we obtain the first statistically significant conclusion on current LLM capabilities for research ideation: we find LLM-generated ideas are judged as more novel (p < 0.05) than human expert ideas while being judged slightly weaker on feasibility. Studying our agent baselines closely, we identify open problems in building and evaluating research agents, including failures of LLM self-evaluation and their lack of diversity in generation. Finally, we acknowledge that human judgements of novelty can be difficult, even by experts, and propose an end-to-end study design which recruits researchers to execute these ideas into full projects, enabling us to study whether these novelty and feasibility judgements result in meaningful differences in research outcome."

AIs generate more novel and exciting research ideas than human experts "The first statistically significant results are in: not only can Large Language Model (LLM) AIs generate new expert-level scientific research ideas, but their ideas are more original and exciting than the best of ours – as judged by human experts."

Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers

Can ChatGPT help researchers understand how the human brain handles language? | PNAS. Really!

I believe the question is possibly too narrow, and short-sighted! Reeks of anthropomorphism!

It appears the author Mitchell Waldrop is a free lance journalist and PhD physicist, not a scientist as you would expect from PNAS.

Machine learning & AI are an avenue to finding new forms of intelligence beyond what evolved over billions of years through natural evolution. It is possibly a quantum leap!

Can ChatGPT help researchers understand how the human brain handles language? | PNAS Large language models are surprisingly good at mimicking our speech and writing. Now they’re serving as an electronic lab rat for language, offering insights into the mysteries of one of humankind’s most important abilities.

Promoting Endangered Languages with AI - IBM Research

Food for though!

To ponder:

1. Would the world not be a better place and more peaceful if we did not speak so many different languages? 
2. Or will machine translation really make us all understand each other better all the time at any time? We'll find out!
3. Will we be able to preserve some of the unwritten wisdom of minority people?

"Nearly half of the world’s 7,000 languages are expected to go extinct by 2100. Now, large language models are being trained to document threatened languages and help encourage their use in everyday life." (Source)

"Today, Nheengatu, which translates to "good language" in English, is down to about 20,000 speakers and is on UNESCO’s list of “severely endangered” languages, meaning it’s no longer being passed down to most children. And it’s not the only language with a diminishing number of speakers. Nearly half of the world’s 7,000 languages are expected to go extinct by 2100. To try and stave off their decline, the United Nations has declared 2022-2032 the International Decade of Indigenous Languages. ...

Working with students at Brazil’s University of Campinas, IBM researchers recently built a prototype for an AI-powered writing assistant in Nheengatu. It’s a language that many students grew up hearing their parents and grandparents speak, but that most never learned to read or write. ..."

Promoting Endangered Languages with AI - IBM Research From about 1600 to 1800, Nheengatu was the lingua franca of the Amazon, one of 800 Indigenous languages spoken during the Portuguese colonization of Brazil. Linguistically, Nheengatu evolved as a way for Indigenous communities to communicate with the Portuguese — and each other.

Newspaper publishers sue Microsoft, OpenAI over copyright infringement

This could become a conundrum for large language models and their voracious appetite for training data!

"KEY POINTS

Newspaper publishers in California, Colorado, Illinois, Florida, Minnesota and New York said that Microsoft and OpenAI used millions of their articles without payment or permission to develop artificial intelligence models for ChatGPT and other products.

The publishers provided examples of ChatGPT and Microsoft’s Copilot chatbot allegedly regurgitating information from their articles without directing people to newspaper websites. ...
The legal challenge comes four months after The New York Times sued OpenAI over copyright infringement in the ChatGPT chatbot that the startup released in late 2022. ..."

Newspaper publishers sue Microsoft, OpenAI over copyright infringement

Researchers at Stanford and Databricks Open-Sourced BioMedLM: A 2.7 Billion Parameter GPT-Style AI Model Trained on PubMed Text

Good news! Interesting approach! More specialized large language models are called for.

Caveat: I did not read the research paper.

What caught my eyes is the excerpt of the research paper captured by my screen print below. I think, the custom tokenization makes a huge difference. Just look how GPT2 mangles words! I bet, this affects model outputs like garbage in, garbage out.

Two of the senior authors of this paper are well known in the ML & AI community: Percy Liang, Christopher D. Manning.

From the abstract:

"Models such as GPT-4 and Med-PaLM 2 have demonstrated impressive performance on a wide variety of biomedical NLP tasks. However, these models have hundreds of billions of parameters, are computationally expensive to run, require users to send their input data over the internet, and are trained on unknown data sources. Can smaller, more targeted models compete? To address this question, we build and release BioMedLM, a 2.7 billion parameter GPT-style autoregressive model trained exclusively on PubMed abstracts and full articles. When fine-tuned, BioMedLM can produce strong multiple-choice biomedical question-answering results competitive with much larger models, such as achieving a score of 57.3% on MedMCQA (dev) and 69.0% on the MMLU Medical Genetics exam. BioMedLM can also be fine-tuned to produce useful answers to patient questions on medical topics. This demonstrates that smaller models can potentially serve as transparent, privacy-preserving, economical and environmentally friendly foundations for particular NLP applications, such as in biomedicine. The model is available on the Hugging Face Hub: this https URL."

Researchers at Stanford and Databricks Open-Sourced BioMedLM: A 2.7 Billion Parameter GPT-Style AI Model Trained on PubMed Text - MarkTechPost

BioMedLM: A 2.7B Parameter Language Model Trained On Biomedical Text (open access)

On OLMo: Accelerating the Science of large Language Models

Good news! Recommendable!

From the abstract:

"... we believe it is essential for the research community to have access to powerful, truly open LMs. To this end, this technical report details the first release of OLMo, a state-of-the-art, truly Open Language Model and its framework to build and study the science of language modeling. Unlike most prior efforts that have only released model weights and inference code, we release OLMo and the whole framework, including training data and training and evaluation code. We hope this release will empower and strengthen the open research community and inspire a new wave of innovation. ..."

[2402.00838] OLMo: Accelerating the Science of Language Models