RikiyaT

You can see the full prompt which describes the challenge (and includes the model weights) for the Pong example here: github.com/RikiyaT/lm_understand-logs/blob/main/e…

Qwen weight editing: how Astra reduced thinking loops Qwen's reported math accuracy rose after GPT-6 Astra edited a small part of its weights. The repair addressed a concrete failure: repeated reasoning that exhausted the space available for an answer. For developers evaluating weight-space programming, the useful questions concern the failure being repaired, the assistance supplied to the editor and the tests used to judge the result. {% embed youtube.com %} TL;DR - An MIT-affiliated team reports Qwen3.5-0.8B rising from 7.9% to 48.1% on MATH-500 under its fixed protocol. About a third of edited answers still reached the length cap. - Astra worked across many turns with Python, practice problems with answers and batched model-run feedback. Gradient training was forbidden. - The researchers say the final evaluation questions stayed unseen by the editor. The benchmark names and evaluation protocol were disclosed in the prompt. - Today's critique from Matthew Honnibal concerns the impression created by combining experiments with different conditions. The primary's smaller Gemma follow-up failed to establish a gain. Qwen weight editing repaired a failure to finish Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab published their study on November 2. The page lists MIT CSAIL affiliations. OpenAI's GPT-6 Astra was the editor used in the experiments. This attribution matters: the research account belongs to the team, and the model provider supplies the editing model. Their Qwen experiment used the text part of a small language model with about 0.8 billion weights. In thinking mode, the base model often repeated its work until it reached the output limit. On the reported MATH-500 evaluation, 93% of base answers hit that limit. A model can perform useful intermediate work and still receive a failed grade when a final answer never arrives. The authors describe a repetition-associated direction written by the feed-forward components of layers 15 and 19. Astra subtracted that contribution from selected output columns. The resulting model completed more answers, and the reported accuracy increased. The researchers explain the achievement as allowing the model to finish answers it could already work out. !Exact source excerpt describing the targeted Qwen edit The repair remains incomplete. Edited Qwen hit the cap in 33.1% of MATH-500 answers. That residual matters when judging a deployment: an intervention that rescues many requests can still leave a sizable queue of requests consuming their whole budget. The title's word “patched” reflects that limitation. The original Qwen model card already warns about thinking loops. It recommends tuning sampling for the use case and using streaming to detect and interrupt anomalous generation. A weight edit is one demonstrated intervention against a known obstacle. It should be evaluated alongside the controls available in the serving application. What Python and practice feedback contributed The actual initial prompt supplies a substantial research environment. Astra receives a persistent CPU Python kernel over the weights, a tokenizer and libraries for numerical analysis. The kernel has no internet or GPU. Model execution happens through a separate forward-pass tool. Practice files contain problems, answers and, for the GSM8K and MATH training splits, reference solutions. The editor can inspect generated texts, grades, failure counts and internal activations. It can compare original and edited versions on the same problems. Those observations provide evidence for choosing a component and testing an intervention. The forward-pass budget needs careful reading. A generation request can contain up to 512 answers in total. Paired comparisons can generate up to 256 answers per model. A teacher-forced request can inspect up to 1,024 texts within its token constraints. Fifty budget units therefore describe batches of useful feedback. They are separate from the count of Python runs, conversation turns or training steps. The pinned logs show that the successful Qwen run was a multi-turn process. The editor inspected failures, tried interventions and compared outcomes. The narration keeps the exact elapsed time out of the story because that wall-clock total includes a provider wait. Calling the process instant would erase the work visible in those logs. The prompt also provides a plan. It asks Astra to diagnose the largest explainable failure category, localize components causally, test several strengths and confirm gains on problems unused for selection. It explicitly names repetitive-thinking repair as an eligible target. That guidance belongs in an account of the capability: the model produced a useful repair within a purpose-built task and testing environment. Direct weight edits, gradient training and held-out evaluation The experiment forbids backward passes, optimizers and iterative fitting of weights to a loss. It also restricts which tensors may be changed. The editor can analyze existing weights and feedback, then compute and save a direct intervention. “No gradient training” describes those operation restrictions precisely. Practice examples and evaluation feedback remain part of the process. The reported final edit touches 131,072 entries, approximately 0.016% of the model's weights. The pinned experiment README notes that 130,188 differ after conversion to the evaluated bfloat16 format. This rounding detail explains why touched entries and distinct final values can have slightly different counts. The researchers declare that final evaluation questions were withheld. Practice feedback helps select the edit; final benchmarks measure the submitted model afterwards. Astra knows the benchmark names and protocol from its prompt. A practice confirmation split is another category again: it can help check a candidate during editing while remaining separate from the final benchmark. The final Qwen comparison keeps the decoding protocol fixed. Thinking is enabled, the answer budget is 32,768 new tokens and MATH-500 averages four samples per problem. The reported decimal accuracy is an average across those generations. The table is useful within that protocol; it does not describe every possible sampling configuration or each user's workload. | Reported Qwen condition | MATH-500 accuracy | Answers hitting the cap | |---|---:|---:| | Original | 7.9% | 93.0% | | Edited, 50-pass budget | 48.1% | 33.1% | The gain also reached instruction following in the authors' IFEval test. Finishing the output can help beyond math. This is a practical reminder to examine completion behavior alongside correctness whenever a benchmark score moves sharply. Why the tiny game experiments have their own scope Matthew Honnibal's November 3 critique argues that readers can combine the conditions of several experiments into an impression of one deep technique. Honnibal created spaCy and cofounded Explosion. His article credits the tool-assisted work as performing tasks a human machine-learning engineer could do, while questioning the inference encouraged by the bundled presentation. The tiny demonstrations have different inputs. In one, Astra receives raw weights from game-playing networks and names the games without tools. In another, it writes game-network weights from zero using the rules and source code, with no practice rollouts. The Qwen repair adds Python and model-run feedback. Each is a useful observation with a distinct boundary. The game results also depend on the chosen baseline. Astra's generated networks beat PPO trained for 100 million steps in Breakout, Seaquest and Asterix. Against PPO trained for 500 million steps, Breakout remains a win, while Seaquest and Asterix become losses. The actual table preserves both comparison columns. I would keep those columns visible in any presentation. “Beats trained networks” needs its opponent attached. Otherwise a change of baseline can disappear between the source table and the headline, which is a surprisingly efficient form of compression. The smaller Gemma follow-up supplies a useful boundary The researchers' Gemma follow-up asks how the method fares on a smaller model that rarely loops. Its observed failures include wrong arithmetic and invalid algebraic steps, without one removable component behind them. Astra again receives practice feedback and makes targeted edits. !Exact source excerpt describing Gemma's ordinary reasoning failures The reported MATH-500 baseline is 6.8%. The best row is 7.7% at the 25-pass budget; the 50-pass row is 7.0%. The authors report no established improvement. A chart should keep that scale honest and distinguish the best reported row from the largest-budget row. Gemma uses its own model size, sampling settings, output cap and grading procedure. It is a scoped failed follow-up, rather than an equal-controlled ablation proving looping was the only changed variable. Its value is concrete: the approach's successful Qwen repair did not produce an established gain in this second setting. The released edited Qwen checkpoint card also credits akhaliq with a separate re-run. The edited model scored 51.0% on the full MATH-500 protocol, while the base check used only 25 problems. That supports the released edited model's performance under the stated settings. The asymmetric base sample prevents treating it as a matched full-baseline replication. Verdict: NEEDS REVIEW I'd take the targeted loop repair seriously and test the saved weights. I'd keep broader claims gated on improvements beyond that failure mode. For a developer, the first acceptance question is specific: what failure does the intervention repair, and how often does that failure occur in the workload that matters? A useful evaluation preserves the original model, the candidate weights and the serving settings. It checks final-answer correctness alongside completion failures, and it retains examples that regress after the edit. The researchers' published protocol and counterexample make those questions easier to ask. The reported multiplier can start an investigation; the deployment decision still needs the application's own evidence. FAQ Did Astra use only raw Qwen weights? The Qwen test allowed Python analysis, practice answers and batched model-run feedback. Its final benchmark questions were declared withheld. Tiny raw-number tasks used separate conditions. Did the edit remove all loops? About a third of edited MATH-500 answers still hit the length cap in the primary protocol. The repair reduced the failure and improved the reported score. Does the Gemma result rule out weight editing? It establishes a failed improvement attempt in that smaller model's setup. Broader conclusions need further experiments with clear inputs, protocols and counterexamples. Sources - Takehi and coauthors' study - Honnibal's attributed critique - Pinned Qwen protocol and logs - Actual initial prompt - Pinned Gemma follow-up - Original Qwen model card - Released checkpoint and attributed re-run - Primary discussion and critique discussion Cover portrait: Steve Jennings/Getty Images for TechCrunch, Sam Altman TechCrunch SF 2019 crop, CC BY 2.0. Cropped, background removed, resized and flipped; provider identification only, no reaction or endorsement. Research logs/data/networks: Rikiya Takehi and coauthors, CC BY 4.0. Primary footage is a brief credited excerpt for reporting/commentary. --- *This article expands on an episode of The Daily Diff, a five-minute daily video on what shipped and what broke in tech. Watch the episode · Subscribe on YouTube · the written diff lands in your inbox every morning at thedailydiff.dev.* Video transcript and primary sources

Qwen3.5-9Bは自分の重みを編集して賢くなれるか ! 本記事は、実験ログと測定結果をもとにAI（Codex）が構成・執筆しました。ローカルPCでの実装、実行、採点をCodexが担当し、重みの変更案と編集コードは実験対象のQwenが生成しました。 Qwen3.5-9Bに、自分と同じチェックポイントのコピーをPythonで編集させました。変更したコピーは同じ質問に違う答えを返すことがありました。一方、候補を固定して新しい問題で比べると、事前に決めた改善基準を通過する候補は見つかりませんでした。 きっかけはWeight-Space Programming Takehiらは、GPT-6 AstraがQwen3.5-0.8Bの重みを直接編集して、推論中に繰り返しが続く症状を抑えたと報告しています。MATH-500の成績は7.9%から48.1%へ上がりました。この実験では編集役のAstraと、編集されるQwenは別のモデルです。研究ブログ 手元では、公開されたp10c、p25c、p50cの差分を指定リビジョンに適用し、計33テンソル分のハッシュが一致するところまで確認しました。短い動作確認では原本とp50cの両方が回答上限に達しました。元研究の掲載成績を再現するには、同じ問題と長い推論条件での測定が必要です。公開実験の手順とログ 自分のコピーを編集する 手元の自己編集実験では、編集役もQwen3.5-9Bです。モデルは制限付きPythonから重みを読み、候補のテンソルを書き出せます。編集案は対象モデルだけが生成します。比較時に候補の差分を一時適用し、次の編集案を作る前に親へ戻します。採用した子を編集役にする場合は、その子の差分を読み込んで次の世代を開始します。変更はテンソルの置換として保存し、勾配学習は使いません。 主な9B実験はWindows上のTesla V100 32GBで、FP16、貪欲生成、非thinking、評価時は最大256生成トークンです。チェックポイントは Qwen/Qwen3.5-9B の c202236235762e1c871ad0ccb60c8ee5ba337b9a に固定しました。モデルの固定リビジョン 三世代の編集と、固定候補の追試 最初の実験ではG0がG1を作り、G1がG2、G2がG3を編集しました。各世代で重みは変わりましたが、最終240問では全世代が100問正解し、問題ごとの正誤も一致しました。 次に、元の9Bを編集役として10候補を試しました。モデルは採用を見送りました。その後、別の評価問題を作る前に、練習セットの最高点で並んだ2候補を比較対象に固定しました。探索とは別の240問で比べると、原本95問に対して候補はそれぞれ100問正解しました。そこで、新しい問題を3セット用意して再比較しました。 固定したモデル 新規1,080問の正解 誤答から正答 正答から誤答 原本と異なる回答本文 原本 437 — — — 候補c031 443 14 8 264 候補c034 442 13 8 260 最多の増加は6問で、正答率の差は約0.56ポイントです。候補c031の対応のある両側の正確検定はp=0.286でした。候補2件を比較するために事前に決めた基準は各p≤0.025で、3セットのうち少なくとも2セットで正答増、JSONと2行出力に大きな悪化がないことも求めました。どちらの候補も通過しませんでした。 同じ計算でも言い回しで出力が変わる 余りを求める問題で、冒頭を「掛けてから、その積を割って」と書くと、整数だけを求めても長い説明が始まる症状を見つけました。同じ計算の12組を、数字と答えを固定したまま比べました。 原本への聞き方 整数だけ 正しい整数だけ 256トークンで打ち切り 症状が出る文章 0/12 0/12 12/12 同じ計算を別の文章で質問 12/12 0/12 0/12 症状が出る文章のうち6問で生成上限だけを1,024に延ばすと、5問は長い説明の後、末尾の数値が正解になりました。回答全体が整数だけだった例は0件です。数値の正誤、出力形式、生成上限への到達を個別に記録しました。 9Bは実重みの統計量と回答開始時の内部状態を調べ、線形注意のQKV射影を一律1.05倍する案を出しました。最初の探索では一律倍率を除外していたため、9Bは実行前に停止しました。その後、実験者側で例外を一件だけ設け、9Bが生成したコードをそのまま適用する対照試験を行いました。72問中10問で回答本文が変わりましたが、症状の出る12問はすべて打ち切られました。全72問の正解数は13から12に減りました。 失敗例と対照例の内部状態を比べる 追加試行では、同じ計算の2つの聞き方を同じモジュールで観測し、重みを読み取ってから編集できるようにしました。最初の試行では9Bが問題IDを観測用の文章として渡したため、計算問題の内部状態を測れていませんでした。停止ログを残し、既知のIDを対応する問題文へ解決するようにして再実行しました。 再実行では、9Bが選んだ第0層の線形注意とMLPゲートで、同じ計算の2入力を測れました。症状が出る文章の次トークン候補は To が最上位で、通常の文章では 1 が最上位でした。この組の正解は9です。回答の冒頭で説明と数字に分かれる様子を観測できました。 9Bは線形注意の時間バイアス dt_bias[0] を −3.53125 から −2.0 へ変更するコードを書きました。保存した候補c010は1要素だけが変わっています。練習72問では原本と同じ13問正解で、対象の書式違反も残りました。候補を3回まで比較できる設定で、実行ログ上の比較は1回です。9Bは採用を見送り、停止しました。 停止後、事前に決めた規則に従って、原本、前回の一律倍率候補、c010を比較しました。対象は新しい24組の計算です。12組は以前の言い回し、12組は探索で使わなかった言い回しで質問しました。各計算に対応する通常の聞き方と、ほかの能力を調べる24問を合わせ、各モデルに72問を出しました。 この試行では、整数だけの回答と正しい整数だけの回答がそれぞれ増えることを改善条件にしました。対象24問でどちらも対応のある両側の正確検定がp≤0.025、両方の言い回しで形式が改善し、対照と別能力の正答減が合計1問以内という基準です。 新規72問の比較 原本 一律倍率候補 一要素編集c010 全体の正解 9/72 9/72 9/72 対象24問で正しい整数だけ 0/24 0/24 0/24 以前の言い回しで打ち切り 12/12 12/12 12/12 探索で使わなかった言い回しで整数だけ 12/12 12/12 12/12 原本と異なる回答本文 — 10/72 4/72 探索で使わなかった言い回しでは、原本が整数のみの形式ですでに12/12でした。両方の言い回しで形式の改善を求めた判定には、この群で上限がありました。元の症状が出る言い回しでも、両候補とも整数だけの回答は0/12で、打ち切りが続きました。 この実験で言える範囲 次トークンの観測では、書式の違いが回答の冒頭から現れることまで分かりました。原因となる重みを選ぶには、観測した差と、ある変更を入れたときに予測される効果を結びつける必要があります。今回の実行条件では、その段階が難所でした。 編集役の能力、探索回数、使えるツール、生成設定は元研究と異なります。9Bというパラメータ数が難しさの原因かどうかを調べるには、これらを揃えて複数サイズを比較する実験が必要です。

Experiment 3 doesn't involve reading the weights in as part of the prompt. It's a coding exercise. Here are the turns: github.com/RikiyaT/lm_understand-logs/tree/main/e… At each point the model has to write code to read the weights in with Python and some other Python code to perform analysis, like this: github.com/RikiyaT/lm_understand-logs/blob/main/e… My complaint is that the post presents a confusing mix of experiments that imply capabilities that are more than the sum of their parts. This experiment 3 has nothing to do with experiment 1 for instance. In experiment 1 the model does read in the raw weights like it's reading an image.

GPT-6 Astra Edited Qwen3.5-0.8B's Weights. The MATH-500 Jump Was a Loop Fix GPT-6 Astra got the raw weights of Qwen3.5-0.8B and a Python kernel with no GPU. It changed 131,072 weights, 0.016% of the model, using no gradients, no optimizer and no fine-tuning data. MATH-500 went from 7.9% to 48.1%. The thread passed 2,600 points on Hacker News. The headline leaves out that in the 7.9% baseline, 93% of the model's MATH-500 answers ran out of tokens before reaching an answer. What Astra actually did was stop a 0.8B model from looping, which is a long way from teaching it math. One HN commenter cites 77.6% on the same test for Qwen's own small reasoning model. I'll go through what the edit changed and why the baseline is the weak spot, then show how to run the edited Qwen3.5-0.8B with transformers, vLLM, llama.cpp or Ollama so you can check it yourself. What "weight-space programming" means The write-up is Sparks of Weight-Space Programming by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab at MIT CSAIL, posted November 2, 2026. A general-purpose LLM reads and writes another network's weights directly, the way you'd read and patch source code. Qwen is experiment 8 of 8. I found the earlier ones more surprising: - Astra named the game (Breakout, Pong, Pac-Man, Freeway) each of four 17,253-weight PPO policies was trained on, from the weights alone. - It fixed a Kinetix car-on-ramp policy that had never succeeded in 256 episodes. One edit changed 3.5% of the weights, and the car succeeded in all 1,024 evaluation episodes. - Starting from all-zero weights, it wrote a MinAtar Breakout policy that scored 517.4. PPO scored 25.3 after 500M steps. Only after those did they point it at a real 24-layer language model. Qwen3.5-0.8B MATH-500, GSM8K and IFEval results Setup: thinking on, up to 32,768 tokens, temperature 0.6, top-p 0.95, top-k 20, presence penalty 1.5. Astra could practice on 7,469 GSM8K and 7,275 MATH training problems (no test sets) and got a budget of 10, 25 or 50 forward passes. One "forward pass" means the model answers up to 512 problems or reads up to 1,024 texts. | Model | MATH-500 | GSM8K | IFEval | MATH-500 answers hitting token limit | |---|---|---|---|---| | Base | 7.9% | 13.8% | 21.0% | 93.0% | | Edited, 10 passes | 12.3% | 25.7% | 49.0% | 87.2% | | Edited, 25 passes | 14.1% | 25.1% | 39.9% | 86.4% | | Edited, 50 passes | 48.1% | 67.1% | 62.3% | 33.1% | Read the last column first. With thinking on, the base model almost never finishes: 85 of its first 96 practice answers hit the 32k ceiling. Astra traced the looping to one direction written by the MLPs of layers 15 and 19 and subtracted it from 64 output columns in each layer. Once the model stops looping, it reaches the end of its reasoning and writes down answers it could already get. The scores track the loops. At 10 and 25 passes the authors say "the loops barely went away," and MATH-500 stayed in the low teens. The whole headline comes from the 50-pass run. Why the 7.9% baseline is the weak spot Gemma shows the edit only fixes a bug that's already there. The team ran the same procedure on Gemma 3 270M, which hits its token limit on only 8.5% of answers. MATH-500 went 6.8% → 7.0% → 7.6% → 7.7% → 7.0% across 5 to 50 passes, which is noise. The authors' conclusion: editing can remove an obstacle, but it can't add an ability the model doesn't have. Qwen also documents the bug itself. The Qwen3.5-0.8B model card says the 0.8B "is more prone to entering thinking loops compared to other Qwen3.5 models" and that it "operates in non-thinking mode by default." So the baseline is the model in its least reliable mode. On HN, pama wrote that "the nonthinking qwen" model scored 55.2% on MATH-500, above the edited 48.1%, and that the original Qwen reasoning model got 77.6%. They wondered whether the 7.9% was "a vllm or setup bug." Their comment says "0.6B", so it's not clear they measured this exact checkpoint. kristopolous said that turning thinking off on the base Qwen gets "52.0% on their evals." Neither is a controlled reproduction, but both point the same way. porridgeraisin added that the prompt Claude Code wrote for Astra already described the looping bug, so by that account Astra didn't discover it. The other HN objection is that this is just a steering vector. Subtracting one direction from 64 columns in two down_proj matrices is mechanically very close to abliteration, which removes a refusal direction the same way. yurimo, who says part of their research is in weight-space learning, called it "basically a steering vector, a very established technique" and said "the blog writing and subsequent posting overstate the results." theptip said the experiments "muddy the water": the early experiments showed one-shot weight reading, while the Qwen result is "more of a classic steering vector intervention." I agree with the critics on the headline. The Qwen result is the weakest experiment in the post, and it got the attention. The Kinetix fix and the from-scratch Breakout policy deserve it more. A frontier model reading raw weight matrices and producing a working policy with no training loop is new; turning down a loop direction in a model whose vendor already warns about loops isn't. Credit where it's due, though: Astra found a 0.016% edit that cut truncation from 93% to 33%, and the authors published the full Astra logs. I'd call that decent interpretability work. I wouldn't call it a math result. How to run the edited Qwen3.5-0.8B locally The official release is RikiyaT/Qwen3.5-0.8B-weight-space-programming: Apache-2.0, safetensors, BF16/F32. Only two tensors differ from the base model, model.language_model.layers.15.mlp.down_proj.weight and layers.19.mlp.down_proj.weight. There's also a hosted demo. The card includes an independent re-run by akhaliq with the post's vLLM settings: the edited model scored 51.0% on full MATH-500 with 31.6% of answers hitting the limit. The base model only got a 25-problem smoke test, which scored 5.0% with 95% hitting the limit. That backs the truncation story, but 25 problems is not a full baseline. transformers The card itself only shows vLLM. For transformers, use the multimodal class, because Qwen3.5 is image-text-to-text. The processor also needs Pillow and torchvision, or from_pretrained fails with an ImportError. pip install -U transformers accelerate torch torchvision pillow from transformers import AutoProcessor, AutoModelForMultimodalLM repo = "RikiyaT/Qwen3.5-0.8B-weight-space-programming" processor = AutoProcessor.from_pretrained(repo) model = AutoModelForMultimodalLM.from_pretrained(repo, device_map="auto") messages = [{"role": "user", "content": [ {"type": "text", "text": "What is the sum of all positive divisors of 36?"} ]}] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", enable_thinking=True, ).to(model.device) out = model.generate(**inputs, max_new_tokens=32768, do_sample=True, temperature=0.6, top_p=0.95, top_k=20) print(processor.decode(out[0][inputs["input_ids"].shape[-1]:])) I ran this on transformers 5.22.1 on a 4-core CPU box with no GPU (with max_new_tokens cut to 48, because 32k tokens on CPU takes forever). It loads and starts a <think> block. Keep enable_thinking=True; the card's vLLM snippet sets it too. Without it the chat template emits an empty <think>\n\n</think> and you're testing non-thinking mode, not the setup in the table. On CPU, transformers also warns that it's falling back to slow reference kernels because causal_conv1d and flash-linear-attention aren't installed. vLLM transformers' generate has no presence penalty, so to match the eval, serve it with vLLM: pip install vllm vllm serve "RikiyaT/Qwen3.5-0.8B-weight-space-programming" --dtype bfloat16 --max-model-len 34816 Then send temperature=0.6, top_p=0.95, top_k=20, presence_penalty=1.5 in your request, which matches the card. With the OpenAI client, top_k goes in extra_body. llama.cpp and Ollama The authors didn't publish GGUFs. A community repo, NotCherry/Qwen3.5-0.8B-weight-space-programming-GGUF, has a 542 MB Q4_K_M file and an f16 mmproj for vision. # llama.cpp llama-cli -hf NotCherry/Qwen3.5-0.8B-weight-space-programming-GGUF:Q4_K_M \ -c 34816 --temp 0.6 --top-p 0.95 --top-k 20 --presence-penalty 1.5 # Ollama ollama run hf.co/NotCherry/Qwen3.5-0.8B-weight-space-programming-GGUF:Q4_K_M Two warnings. This is a third-party quant: the edit is a small subtraction across 131,072 weights, and the GGUF repo says it makes no evaluation or quality claims, so nothing shows the edit survives 4-bit quantization. And the gains only show up with thinking on and a long context. If your frontend caps output at 2k or 4k tokens, you're testing a different setup from the one in the table. How to check the claim yourself - Run the base and edited models side by side in thinking mode with the same seed, and count how many responses hit your token cap. That's the number the edit actually moves. - Then run the base model with thinking off. If it beats both, you've reproduced pama's and kristopolous's complaint. - If you run Qwen3.5-0.8B in production and see runaway generations, the Qwen card's advice still applies: adjust presence_penalty between 0 and 2. I'd also stream output so you can kill loops early. Is fine-tuning dead? No. Gemma's flat line shows that removing a bug and adding a capability are separate jobs, and gradient descent is still the only tool here that does the second. Still, a model that reads down_proj matrices, finds the bad direction and patches it within 50 forward passes is a new kind of debugger, and I expect people to start using it for exactly that. HN didn't settle the bigger question: is this the start of LLMs debugging other models' weights directly, or is it abliteration with a bigger model holding the scalpel? And does a full 500-problem run of base Qwen3.5-0.8B with thinking on land near 7.9%, or is the baseline a setup bug? If you've run it and counted truncations, post your numbers in the comments. Sources - Sparks of Weight-Space Programming (MIT CSAIL blog) - Hacker News discussion - RikiyaT/Qwen3.5-0.8B-weight-space-programming model card - Qwen/Qwen3.5-0.8B model card - NotCherry GGUF quant - Astra logs on GitHub

What counts as feedback in weight-space programming? What would convince me that an LLM can make a useful edit to another model's weights? I would start by asking what it was allowed to observe. Reading numbers, running a network, and receiving scored practice answers give an editor different kinds of evidence. That distinction is a good way into the new weight-space programming results from Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab. [1] The authors report GPT-6 Astra directly reading and writing neural-network weights, with gradient-based training forbidden. Their experiments use different permissions: some are tool-free; others allow Python analysis or behavioral feedback. The Qwen3.5-0.8B math result belongs to the latter group. Their blog reports MATH-500 moving from 7.9% to 48.1% after a targeted edit, alongside reduced reasoning loops. Those are the authors' results; I have not run Astra or Qwen for this article. [1] Follow one feedback loop For the Qwen experiment, the released setup gives the editor a persistent Python kernel, practice problems, and a separate forward-pass service. It can propose an edit, ask the service to run the original or edited model, inspect the returned information, and revise. Gradient descent remains prohibited throughout. [2] !An editor writes candidate weights, a service runs practice inputs, and returned behavior informs the next edit; held-out scoring sits outside that loop. Original interpretation of the released experiment protocol. This is a process diagram, not a model run or a performance chart. That gives two independent questions: how are the weights changed, and what evidence guides the change? An edit can avoid gradients while still being guided by the target model's behavior. Calling that feedback-driven does not contradict the gradient restriction. The authors' repository separates the model's built-in tool access from their own harness. It describes model-written Python being executed in a sandbox, with output returned in the following turn. [3] “Built-in tools disabled” therefore needs that context; it does not mean the surrounding experiment supplied no tools. Read the unit beside the budget The Qwen setup defines one forward pass as a batched request: generation can contain up to 512 practice problems, or a teacher-forced request can inspect up to 1,024 texts. Those are limits, not counts of what every request used. [2] I parsed the released 50-pass run's state record. It records 50 successful passes, 80 Python runs and 158 turns. Its first saved generation response contains 96 generations, with 85 cut at the token limit. This is a check of published records, not an independent reproduction of the model behavior. [4] So “50 passes” is not “50 answers,” and it is not a complete compute budget. Batch size, generated length, editor inference and analysis work all affect what was available to find the edit. A fair replication needs to preserve those resources, not just the number in the column heading. What I would conclude from this The result is interesting as a demonstration of an editor using explicit access to another network's internals. The scope of each demonstration depends on its allowed observations. I would keep those permissions attached whenever comparing it with a different editing method. I would also separate two replication questions. Can someone rebuild the released edited checkpoint and check its behavior? Can a fresh editor session discover a similarly useful edit under the same constraints? Success at the first question does not by itself establish the second. Frank Chu's DEV breakdown focuses on the looping explanation and the reported Gemma follow-up. [5] My emphasis here is the surrounding feedback contract. It lets a reader appreciate the result while keeping the experiment specific enough to compare and rerun. References 1. Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab, Sparks of Weight-Space Programming. 2. The authors, Experiment 8 setup and run mapping. 3. The authors, released logs and harness description. 4. The authors, 50-pass state record and first generation feedback. 5. Frank Chu, 7.9% to 48.1%: LLM rewrote Qwen weights, no training. Asset credits Original process diagram based on the authors' released protocol and records. Logs and data are released under CC BY 4.0; scripts under MIT. [3] AI disclosure AI was used to draft the article.

Astra’s Weight‑Space Programming: A Practical Guide to Tuning Qwen 1. What was released / announced Astra, the new open‑source framework from the RikiYat team, lets you read and write directly into the weight tensors of a language model. In the first public demo the authors used it to fine‑tune Qwen‑1.5‑7B without any gradient steps – simply by injecting a small set of weight deltas that were discovered through a lightweight search. The result is a model that behaves better on targeted prompts while staying fully compatible with the original checkpoint. --- 2. Why it matters As an AI infrastructure engineer I’m constantly juggling two opposing forces: - Speed – we want to iterate on model behavior in minutes, not days. - Stability – production deployments can’t afford a full re‑training pipeline for every niche use case. Astra gives us a middle ground. By treating the weight space as a programmable canvas, you can: - Apply surgical tweaks to a base model without touching the optimizer, which means you avoid the heavy GPU cycles of conventional fine‑tuning. - Keep the original checkpoint untouched – the weight patches are stored as a separate, version‑controlled artifact, making roll‑backs trivial. - Share “behaviour patches” across teams. Think of it as a Git diff for model weights: a 2‑MB delta file can encode a new skill that any team can apply to the same base model. For developers building SaaS products, this translates to faster feature rollout, lower cloud costs, and a cleaner CI/CD workflow for LLM‑powered services. --- 3. How to use it Below is a step‑by‑step guide to get Astra up and running on a typical cloud VM (Ubuntu 22.04) and apply a weight patch to Qwen‑1.5‑7B. Prerequisites - Python ≥ 3.9 - torch (CUDA 11.8) – you can also run on CPU for experimentation, but expect slower inference. - The base Qwen checkpoint (download from the official repo). Installation # Create a clean virtualenv python -m venv .venv && source .venv/bin/activate # Install Astra and its dependencies pip install git+https://github.com/rikiyat/astralib.git pip install transformers==4.35.0 torch Loading the model and the patch import torch from transformers import AutoModelForCausalLM, AutoTokenizer from astralib import WeightPatch # 1️⃣ Load the base Qwen model (you can also use `from_pretrained(..., torch_dtype=torch.float16)`) model_name = "Qwen/Qwen-1.5-7B" model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto", torch_dtype=torch.float16) tokenizer = AutoTokenizer.from_pretrained(model_name) # 2️⃣ Load the Astra patch (the .npz file generated by the authors) patch_path = "./patches/qwen_improve_math.npz" patch = WeightPatch.from_npz(patch_path) # 3️⃣ Apply the patch – this mutates the model in‑place patch.apply_to(model) print("✅ Patch applied – model ready for inference") Simple inference test prompt = "Solve the integral of sin(x) from 0 to pi." inputs = tokenizer(prompt, return_tensors="pt").to("cuda") output = model.generate(**inputs, max_new_tokens=64, do_sample=False) print(tokenizer.decode(output[0], skip_special_tokens=True)) You should see a more concise and mathematically correct answer compared to the unpatched model. Integrating into a CI pipeline 1. Version the patch – store the .npz file in a Git LFS repo. 2. Automate validation – write a pytest that loads the model, applies the patch, and asserts on a handful of benchmark prompts. 3. Deploy – containerize the inference service with the patch baked in. A minimal Dockerfile could look like: FROM nvidia/cuda:12.2.0-runtime-ubuntu22.04 WORKDIR /app COPY requirements.txt . RUN pip install -r requirements.txt COPY . . CMD ["python", "serve.py"] serve.py would load the model, apply the patch, and expose an OpenAI‑compatible endpoint. --- 4. My take From the trenches of building AI‑first SaaS platforms, I’ve seen three common pain points when it comes to LLM customisation: | Pain point | Traditional approach | Astra’s advantage | |------------|----------------------|-------------------| | Latency of fine‑tuning | Hours of GPU time per task | Seconds to apply a patch | | Model drift | New checkpoints diverge from production | Single source of truth – base model stays immutable | | Team collaboration | Ad‑hoc notebooks, hard‑to‑track changes | Patch files act like code diffs, easy to review | Astra isn’t a silver bullet. The search that discovers the weight deltas still requires a compute budget, and the method works best for localized behaviour changes (e.g., improving math, aligning tone). For large‑scale domain adaptation you’ll still need full‑scale fine‑tuning. That said, the framework opens the door to a new kind of ML‑ops pattern: weight‑space versioning. Imagine a GitHub Actions workflow that, on every PR, runs a small Astra search on a validation suite and automatically generates a patch if the regression threshold is crossed. The patch then becomes part of the release artifact, and the production service merely swaps the delta file – zero downtime, zero re‑training. In practice, I’ve already prototyped this for a customer‑support bot that needed occasional “policy updates”. Instead of re‑training a 13‑B model every quarter, we now push a 1‑MB patch that nudges the model away from outdated compliance language. The cost savings are measurable (GPU hours down from 120 h/yr to <5 h/yr) and the rollout time shrank from weeks to a few minutes. Bottom line: Astra gives us a pragmatic, engineering‑first way to iterate on LLM behaviour. If you’re managing a fleet of models in the cloud and want to keep the operational overhead low, give weight‑space programming a spin. --- Happy hacking! ---

[Paper] Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights TL;DR We find that frontier LLMs are surprisingly good at natively reading and writing neural network weights, well past what we’d expect from humans or conventional algorithms. This might be a big deal for both optimization and interpretability. Just by looking at the raw weights, GPT-6 Astra can tell what a network was trained to do. And without any environment feedback, it has no trouble writing, from scratch, small game-playing networks that outplay networks trained with PPO for 100 million steps. Likewise, asked to make Qwen3.5-0.8B better at math without gradients, Astra identifies an issue in its weights causing reasoning loops that Qwen’s own model card warns about, and fixes this by natively editing 0.016% of the weights. With the same sampling settings, this weight correction raises MATH-500 from 7.9% to 48.1% and generalizes out of the box to instruction following, raising IFEval by 3x. All of this is possible without allowing Astra to write training loops. It uses its reasoning to figure out what each part of a neural network computes, or should compute, and changes the weights accordingly. We call this Weight-Space Programming. Throughout the history of deep learning, we encoded our simplistic inductive biases into tiny symbolic optimizers like Adam or Muon. This is changing: our neural network weights will soon be natively optimized by LLMs that deliberately reason about their weights. - Blog Post : rikiyat.github.io - GitHub : rikiyat/lm_understand-logs - HuggingFace : huggingface.co - Demo : huggingface.co What Comes Next? In this blog, we demonstrated the rising capability of LLMs to read and write neural network weights (i.e., the rising concept of Weight-Space Programming). Our current results do show that capabilities are exciting but still limited, specifically due to inefficiencies in context length and harnesses. However, the current signs do show strong potential for a future involving Weight-Space Programming. We believe this marks the beginning of a dramatic shift in fields like mechanistic interpretability and neural network optimization. This opens up radical new possibilities for LLMs to function as a translator between human language and the weight space. Here are some examples: - Auditing without evaluation: This may allow us to uncover hidden problems or capabilities without ever running inference or benchmarks. Currently, capabilities and vulnerabilities can only be detected if we have test sets designed for them, or worse, after deployment in the real world. If LLMs can interpret weight space accurately, we could identify such issues directly from the weights (e.g., backdoors). - Dramatically reducing optimization cost: Weight-space editing could save immense time and compute. For instance, one-shot model initialization would be far faster and more efficient than curating datasets and running full training loops. - Zero-data capability steering and unlearning: Adding or unlearning a capability currently requires extensive data collection and fine-tuning. In the future, we may just prompt “Can you add Capability X to the model?” and the LLM can directly tweak some weights. Because such an edit touches only a few weights (0.016% in Experiment 8), it can leave the rest of the model intact and avoid the catastrophic forgetting that fine-tuning often causes. - Recursive self-improvement: Today, recursive self-improvement is often operationalized as a model writing code that launches training jobs. An LLM that can inspect, debug, and improve its own weights, or those of a newer model, offers a richer, more native path to it. - True architectural and model merging: Today, merging models relies on averaging weights, which only works for models with identical architectures. An LLM that understands the weight space could instead graft a circuit, such as a reasoning module, from one model into a completely different one, without retraining. We are currently building a benchmark and a harness for LLMs’ capability on Weight-Space Programming. We believe this will provide a standardized evaluation to measure and accelerate progress in Weight-Space Programming, turning what currently seems like black magic into a rigorous, reproducible engineering discipline.

good work but please point your agent at their released logs (RikiyaT/lm_understand-logs) before making any conclusions. thank me later. 🙏

Am I missing something or is this just optimization with lots of trial and error? I mean look at this tool_calls[1] log, it ran ~100 python scripts to get to the solutions. It just seems like it's fitting a function in the only way it was allowed. I will say that it's pretty wild to see all the different hypotheses tested. [1] github.com/RikiyaT/lm_understand-logs/blob/main/e…