GPT · rikiyat.github.io
Astra can read and write weights to improve Qwen
Compiled by KHAO Editorial — aggregated from 1 source. See llms.txt for citation guidance.
◌ Single Source
The team find that frontier LLMs are surprisingly good at natively reading and writing neural network weights, well past what they'd expect from humans or conventional algorithms.
Key facts
- With a budget of 0 forward passes, it wins more than half of its games against two smaller PPO networks (243 → 32 → 32 → 82)
- Likewise, asked to make Qwen3.5-0.8B better at math without gradients, Astra identifies an issue in its weights causing reasoning loops that Qwen’s own model card warns about, and fixes
- Specifically, they test it from a budget of 0 forward passes, all the way to 50 forward passes (each forward pass has a batch of 128 rollouts)
- By looking at the raw weights, GPT-6 Astra can tell what a network was trained
Summary
By looking at the raw weights, GPT-6 Astra can tell what a network was trained to do. Likewise, asked to make Qwen3.5-0.8B better at math without gradients, Astra identifies an issue in its weights causing reasoning loops that Qwen’s own model card warns about, and fixes this by natively editing 0.016% of the weights. All of this is possible without allowing Astra to write training loops. Throughout the history of deep learning, they encoded their simplistic inductive biases into tiny symbolic optimizers like Adam or Muon.