The Astra Qwen Jump From 7.9% to 48.1% Needed Python
A rebuttal shows the Astra post's Qwen maths fix and its PPO-beating networks come from written Python programs, while raw-weight reading happens in separate, smaller experiments.
The Astra post’s abstract sells a single trick: GPT-6 reads raw weights, then improves Qwen3.5-0.8B, no gradients, no environment feedback. Its first line is ‘Just by looking at the raw weights, GPT-6 Astra can tell what a network was trained to do.’ Read alone, that is a modest inspection claim. Read beside 7.9% to 48.1% on MATH-500, it sounds like one capability doing both jobs.
It is two. Two experiments only read raw weights: a 17K-parameter PPO network whose game it names 4/4 times, and a 24K diffusion network whose output shape it names 3/3 times. Neither edits weights, runs Python, nor runs the network. Both are pure inspection, nothing more. A 7.2M character transformer is the reverse: it writes Python and finds language and copying heads without reading raw weights.
The Qwen maths fix and the game-playing networks that outplay PPO-trained nets sit in the Python group; they are what the abstract advertises. The stage-magic comparison is apt. A magician does the trick with a gimmicked prop, then without it, then with a forced choice, so the audience assumes one technique and each performance reads as a disproof. No single experiment carries the whole load.
The individual results may hold: if the 0.016% edit reproduces, that is real weight surgery and someone should try it. The attribution is what misleads. Inspection and program search carry different claims and different budgets, and the gap is practical, not pedantic; it decides what gets funded next. Skip it if the abstract alone fills the slide; otherwise read the ‘Python?’ column of the experiment table first.