Large language models (LLMs) can edit other LLMs for improvement and optimization

Infinite improvement by recursion comes to mind!

"Research suggests that you can improve a neural network’s weights simply by prompting an LLM — for networks of less than 1 billion parameters, at least.

What’s new: ... showed that GPT-6 Astra can interpret, improve, and produce neural network weights in proof-of-concept studies.

How it works: The authors asked GPT-6 Astra to
(a) identify which game a network was trained on based on its weights,
(b) alter the weights of a network to improve its ability to operate a simulated car, and
(c) generate a network that would win a game. In a more practical experiment, they asked GPT-6 Astra to improve Qwen3.5-0.8B’s math performance. ..."

"TL;DR

We find that frontier LLMs are surprisingly good at natively reading and writing neural network weights much like code, sometimes with tools such as a Python kernel and sometimes from the raw weights alone. These results show sparks of Weight-Space Programming, which may extend beyond what we’d expect from humans or conventional algorithms. This might be a big deal for both optimization and interpretability.

Just by looking at the raw weights, GPT-6 Astra can tell what a network was trained to do. And without any environment feedback, it has no trouble writing, from scratch, small game-playing networks that outplay networks trained with PPO for 100 million steps.

Likewise, asked to make Qwen3.5-0.8B better at math without gradients, Astra identifies an issue in its weights causing reasoning loops that Qwen’s own model card warns about, and fixes this by natively editing 0.016% of the weights.

With the same sampling settings, this weight correction raises MATH-500 from 7.9% to 48.1% and generalizes out of the box to instruction following, raising IFEval by 3x.

All of this is possible without allowing Astra to write training loops. It uses its reasoning to figure out what each part of a neural network computes, or should compute, and changes the weights accordingly, much as a programmer reads and edits code.

We call this concept Weight-Space Programming: treating the weights as a program to be read and written.

Throughout the history of deep learning, we encoded our simplistic inductive biases into tiny symbolic optimizers like Adam or Muon.

This is changing: our neural network weights will soon be natively optimized by LLMs that deliberately reason about their weights.

..."

GPT-6 Cyber Versus Linux, The Problem With LLM-Powered Search, LLMs Edit Model Weights, Agentic Doc Retrieval

LLMs can natively read & write neural network weights