rikiyat

GPT-6 Astra Edited Qwen3.5-0.8B's Weights. The MATH-500 Jump Was a Loop Fix GPT-6 Astra got the raw weights of Qwen3.5-0.8B and a Python kernel with no GPU. It changed 131,072 weights, 0.016% of the model, using no gradients, no optimizer and no fine-tuning data. MATH-500 went from 7.9% to 48.1%. The thread passed 2,600 points on Hacker News. The headline leaves out that in the 7.9% baseline, 93% of the model's MATH-500 answers ran out of tokens before reaching an answer. What Astra actually did was stop a 0.8B model from looping, which is a long way from teaching it math. One HN commenter cites 77.6% on the same test for Qwen's own small reasoning model. I'll go through what the edit changed and why the baseline is the weak spot, then show how to run the edited Qwen3.5-0.8B with transformers, vLLM, llama.cpp or Ollama so you can check it yourself. What "weight-space programming" means The write-up is Sparks of Weight-Space Programming by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab at MIT CSAIL, posted November 2, 2026. A general-purpose LLM reads and writes another network's weights directly, the way you'd read and patch source code. Qwen is experiment 8 of 8. I found the earlier ones more surprising: - Astra named the game (Breakout, Pong, Pac-Man, Freeway) each of four 17,253-weight PPO policies was trained on, from the weights alone. - It fixed a Kinetix car-on-ramp policy that had never succeeded in 256 episodes. One edit changed 3.5% of the weights, and the car succeeded in all 1,024 evaluation episodes. - Starting from all-zero weights, it wrote a MinAtar Breakout policy that scored 517.4. PPO scored 25.3 after 500M steps. Only after those did they point it at a real 24-layer language model. Qwen3.5-0.8B MATH-500, GSM8K and IFEval results Setup: thinking on, up to 32,768 tokens, temperature 0.6, top-p 0.95, top-k 20, presence penalty 1.5. Astra could practice on 7,469 GSM8K and 7,275 MATH training problems (no test sets) and got a budget of 10, 25 or 50 forward passes. One "forward pass" means the model answers up to 512 problems or reads up to 1,024 texts. | Model | MATH-500 | GSM8K | IFEval | MATH-500 answers hitting token limit | |---|---|---|---|---| | Base | 7.9% | 13.8% | 21.0% | 93.0% | | Edited, 10 passes | 12.3% | 25.7% | 49.0% | 87.2% | | Edited, 25 passes | 14.1% | 25.1% | 39.9% | 86.4% | | Edited, 50 passes | 48.1% | 67.1% | 62.3% | 33.1% | Read the last column first. With thinking on, the base model almost never finishes: 85 of its first 96 practice answers hit the 32k ceiling. Astra traced the looping to one direction written by the MLPs of layers 15 and 19 and subtracted it from 64 output columns in each layer. Once the model stops looping, it reaches the end of its reasoning and writes down answers it could already get. The scores track the loops. At 10 and 25 passes the authors say "the loops barely went away," and MATH-500 stayed in the low teens. The whole headline comes from the 50-pass run. Why the 7.9% baseline is the weak spot Gemma shows the edit only fixes a bug that's already there. The team ran the same procedure on Gemma 3 270M, which hits its token limit on only 8.5% of answers. MATH-500 went 6.8% → 7.0% → 7.6% → 7.7% → 7.0% across 5 to 50 passes, which is noise. The authors' conclusion: editing can remove an obstacle, but it can't add an ability the model doesn't have. Qwen also documents the bug itself. The Qwen3.5-0.8B model card says the 0.8B "is more prone to entering thinking loops compared to other Qwen3.5 models" and that it "operates in non-thinking mode by default." So the baseline is the model in its least reliable mode. On HN, pama wrote that "the nonthinking qwen" model scored 55.2% on MATH-500, above the edited 48.1%, and that the original Qwen reasoning model got 77.6%. They wondered whether the 7.9% was "a vllm or setup bug." Their comment says "0.6B", so it's not clear they measured this exact checkpoint. kristopolous said that turning thinking off on the base Qwen gets "52.0% on their evals." Neither is a controlled reproduction, but both point the same way. porridgeraisin added that the prompt Claude Code wrote for Astra already described the looping bug, so by that account Astra didn't discover it. The other HN objection is that this is just a steering vector. Subtracting one direction from 64 columns in two down_proj matrices is mechanically very close to abliteration, which removes a refusal direction the same way. yurimo, who says part of their research is in weight-space learning, called it "basically a steering vector, a very established technique" and said "the blog writing and subsequent posting overstate the results." theptip said the experiments "muddy the water": the early experiments showed one-shot weight reading, while the Qwen result is "more of a classic steering vector intervention." I agree with the critics on the headline. The Qwen result is the weakest experiment in the post, and it got the attention. The Kinetix fix and the from-scratch Breakout policy deserve it more. A frontier model reading raw weight matrices and producing a working policy with no training loop is new; turning down a loop direction in a model whose vendor already warns about loops isn't. Credit where it's due, though: Astra found a 0.016% edit that cut truncation from 93% to 33%, and the authors published the full Astra logs. I'd call that decent interpretability work. I wouldn't call it a math result. How to run the edited Qwen3.5-0.8B locally The official release is RikiyaT/Qwen3.5-0.8B-weight-space-programming: Apache-2.0, safetensors, BF16/F32. Only two tensors differ from the base model, model.language_model.layers.15.mlp.down_proj.weight and layers.19.mlp.down_proj.weight. There's also a hosted demo. The card includes an independent re-run by akhaliq with the post's vLLM settings: the edited model scored 51.0% on full MATH-500 with 31.6% of answers hitting the limit. The base model only got a 25-problem smoke test, which scored 5.0% with 95% hitting the limit. That backs the truncation story, but 25 problems is not a full baseline. transformers The card itself only shows vLLM. For transformers, use the multimodal class, because Qwen3.5 is image-text-to-text. The processor also needs Pillow and torchvision, or from_pretrained fails with an ImportError. pip install -U transformers accelerate torch torchvision pillow from transformers import AutoProcessor, AutoModelForMultimodalLM repo = "RikiyaT/Qwen3.5-0.8B-weight-space-programming" processor = AutoProcessor.from_pretrained(repo) model = AutoModelForMultimodalLM.from_pretrained(repo, device_map="auto") messages = [{"role": "user", "content": [ {"type": "text", "text": "What is the sum of all positive divisors of 36?"} ]}] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", enable_thinking=True, ).to(model.device) out = model.generate(**inputs, max_new_tokens=32768, do_sample=True, temperature=0.6, top_p=0.95, top_k=20) print(processor.decode(out[0][inputs["input_ids"].shape[-1]:])) I ran this on transformers 5.22.1 on a 4-core CPU box with no GPU (with max_new_tokens cut to 48, because 32k tokens on CPU takes forever). It loads and starts a <think> block. Keep enable_thinking=True; the card's vLLM snippet sets it too. Without it the chat template emits an empty <think>\n\n</think> and you're testing non-thinking mode, not the setup in the table. On CPU, transformers also warns that it's falling back to slow reference kernels because causal_conv1d and flash-linear-attention aren't installed. vLLM transformers' generate has no presence penalty, so to match the eval, serve it with vLLM: pip install vllm vllm serve "RikiyaT/Qwen3.5-0.8B-weight-space-programming" --dtype bfloat16 --max-model-len 34816 Then send temperature=0.6, top_p=0.95, top_k=20, presence_penalty=1.5 in your request, which matches the card. With the OpenAI client, top_k goes in extra_body. llama.cpp and Ollama The authors didn't publish GGUFs. A community repo, NotCherry/Qwen3.5-0.8B-weight-space-programming-GGUF, has a 542 MB Q4_K_M file and an f16 mmproj for vision. # llama.cpp llama-cli -hf NotCherry/Qwen3.5-0.8B-weight-space-programming-GGUF:Q4_K_M \ -c 34816 --temp 0.6 --top-p 0.95 --top-k 20 --presence-penalty 1.5 # Ollama ollama run hf.co/NotCherry/Qwen3.5-0.8B-weight-space-programming-GGUF:Q4_K_M Two warnings. This is a third-party quant: the edit is a small subtraction across 131,072 weights, and the GGUF repo says it makes no evaluation or quality claims, so nothing shows the edit survives 4-bit quantization. And the gains only show up with thinking on and a long context. If your frontend caps output at 2k or 4k tokens, you're testing a different setup from the one in the table. How to check the claim yourself - Run the base and edited models side by side in thinking mode with the same seed, and count how many responses hit your token cap. That's the number the edit actually moves. - Then run the base model with thinking off. If it beats both, you've reproduced pama's and kristopolous's complaint. - If you run Qwen3.5-0.8B in production and see runaway generations, the Qwen card's advice still applies: adjust presence_penalty between 0 and 2. I'd also stream output so you can kill loops early. Is fine-tuning dead? No. Gemma's flat line shows that removing a bug and adding a capability are separate jobs, and gradient descent is still the only tool here that does the second. Still, a model that reads down_proj matrices, finds the bad direction and patches it within 50 forward passes is a new kind of debugger, and I expect people to start using it for exactly that. HN didn't settle the bigger question: is this the start of LLMs debugging other models' weights directly, or is it abliteration with a bigger model holding the scalpel? And does a full 500-problem run of base Qwen3.5-0.8B with thinking on land near 7.9%, or is the baseline a setup bug? If you've run it and counted truncations, post your numbers in the comments. Sources - Sparks of Weight-Space Programming (MIT CSAIL blog) - Hacker News discussion - RikiyaT/Qwen3.5-0.8B-weight-space-programming model card - Qwen/Qwen3.5-0.8B model card - NotCherry GGUF quant - Astra logs on GitHub

What counts as feedback in weight-space programming? What would convince me that an LLM can make a useful edit to another model's weights? I would start by asking what it was allowed to observe. Reading numbers, running a network, and receiving scored practice answers give an editor different kinds of evidence. That distinction is a good way into the new weight-space programming results from Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab. [1] The authors report GPT-6 Astra directly reading and writing neural-network weights, with gradient-based training forbidden. Their experiments use different permissions: some are tool-free; others allow Python analysis or behavioral feedback. The Qwen3.5-0.8B math result belongs to the latter group. Their blog reports MATH-500 moving from 7.9% to 48.1% after a targeted edit, alongside reduced reasoning loops. Those are the authors' results; I have not run Astra or Qwen for this article. [1] Follow one feedback loop For the Qwen experiment, the released setup gives the editor a persistent Python kernel, practice problems, and a separate forward-pass service. It can propose an edit, ask the service to run the original or edited model, inspect the returned information, and revise. Gradient descent remains prohibited throughout. [2] !An editor writes candidate weights, a service runs practice inputs, and returned behavior informs the next edit; held-out scoring sits outside that loop. Original interpretation of the released experiment protocol. This is a process diagram, not a model run or a performance chart. That gives two independent questions: how are the weights changed, and what evidence guides the change? An edit can avoid gradients while still being guided by the target model's behavior. Calling that feedback-driven does not contradict the gradient restriction. The authors' repository separates the model's built-in tool access from their own harness. It describes model-written Python being executed in a sandbox, with output returned in the following turn. [3] “Built-in tools disabled” therefore needs that context; it does not mean the surrounding experiment supplied no tools. Read the unit beside the budget The Qwen setup defines one forward pass as a batched request: generation can contain up to 512 practice problems, or a teacher-forced request can inspect up to 1,024 texts. Those are limits, not counts of what every request used. [2] I parsed the released 50-pass run's state record. It records 50 successful passes, 80 Python runs and 158 turns. Its first saved generation response contains 96 generations, with 85 cut at the token limit. This is a check of published records, not an independent reproduction of the model behavior. [4] So “50 passes” is not “50 answers,” and it is not a complete compute budget. Batch size, generated length, editor inference and analysis work all affect what was available to find the edit. A fair replication needs to preserve those resources, not just the number in the column heading. What I would conclude from this The result is interesting as a demonstration of an editor using explicit access to another network's internals. The scope of each demonstration depends on its allowed observations. I would keep those permissions attached whenever comparing it with a different editing method. I would also separate two replication questions. Can someone rebuild the released edited checkpoint and check its behavior? Can a fresh editor session discover a similarly useful edit under the same constraints? Success at the first question does not by itself establish the second. Frank Chu's DEV breakdown focuses on the looping explanation and the reported Gemma follow-up. [5] My emphasis here is the surrounding feedback contract. It lets a reader appreciate the result while keeping the experiment specific enough to compare and rerun. References 1. Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab, Sparks of Weight-Space Programming. 2. The authors, Experiment 8 setup and run mapping. 3. The authors, released logs and harness description. 4. The authors, 50-pass state record and first generation feedback. 5. Frank Chu, 7.9% to 48.1%: LLM rewrote Qwen weights, no training. Asset credits Original process diagram based on the authors' released protocol and records. Logs and data are released under CC BY 4.0; scripts under MIT. [3] AI disclosure AI was used to draft the article.

Astra’s Weight‑Space Programming: A Practical Guide to Tuning Qwen 1. What was released / announced Astra, the new open‑source framework from the RikiYat team, lets you read and write directly into the weight tensors of a language model. In the first public demo the authors used it to fine‑tune Qwen‑1.5‑7B without any gradient steps – simply by injecting a small set of weight deltas that were discovered through a lightweight search. The result is a model that behaves better on targeted prompts while staying fully compatible with the original checkpoint. --- 2. Why it matters As an AI infrastructure engineer I’m constantly juggling two opposing forces: - Speed – we want to iterate on model behavior in minutes, not days. - Stability – production deployments can’t afford a full re‑training pipeline for every niche use case. Astra gives us a middle ground. By treating the weight space as a programmable canvas, you can: - Apply surgical tweaks to a base model without touching the optimizer, which means you avoid the heavy GPU cycles of conventional fine‑tuning. - Keep the original checkpoint untouched – the weight patches are stored as a separate, version‑controlled artifact, making roll‑backs trivial. - Share “behaviour patches” across teams. Think of it as a Git diff for model weights: a 2‑MB delta file can encode a new skill that any team can apply to the same base model. For developers building SaaS products, this translates to faster feature rollout, lower cloud costs, and a cleaner CI/CD workflow for LLM‑powered services. --- 3. How to use it Below is a step‑by‑step guide to get Astra up and running on a typical cloud VM (Ubuntu 22.04) and apply a weight patch to Qwen‑1.5‑7B. Prerequisites - Python ≥ 3.9 - torch (CUDA 11.8) – you can also run on CPU for experimentation, but expect slower inference. - The base Qwen checkpoint (download from the official repo). Installation # Create a clean virtualenv python -m venv .venv && source .venv/bin/activate # Install Astra and its dependencies pip install git+https://github.com/rikiyat/astralib.git pip install transformers==4.35.0 torch Loading the model and the patch import torch from transformers import AutoModelForCausalLM, AutoTokenizer from astralib import WeightPatch # 1️⃣ Load the base Qwen model (you can also use `from_pretrained(..., torch_dtype=torch.float16)`) model_name = "Qwen/Qwen-1.5-7B" model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto", torch_dtype=torch.float16) tokenizer = AutoTokenizer.from_pretrained(model_name) # 2️⃣ Load the Astra patch (the .npz file generated by the authors) patch_path = "./patches/qwen_improve_math.npz" patch = WeightPatch.from_npz(patch_path) # 3️⃣ Apply the patch – this mutates the model in‑place patch.apply_to(model) print("✅ Patch applied – model ready for inference") Simple inference test prompt = "Solve the integral of sin(x) from 0 to pi." inputs = tokenizer(prompt, return_tensors="pt").to("cuda") output = model.generate(**inputs, max_new_tokens=64, do_sample=False) print(tokenizer.decode(output[0], skip_special_tokens=True)) You should see a more concise and mathematically correct answer compared to the unpatched model. Integrating into a CI pipeline 1. Version the patch – store the .npz file in a Git LFS repo. 2. Automate validation – write a pytest that loads the model, applies the patch, and asserts on a handful of benchmark prompts. 3. Deploy – containerize the inference service with the patch baked in. A minimal Dockerfile could look like: FROM nvidia/cuda:12.2.0-runtime-ubuntu22.04 WORKDIR /app COPY requirements.txt . RUN pip install -r requirements.txt COPY . . CMD ["python", "serve.py"] serve.py would load the model, apply the patch, and expose an OpenAI‑compatible endpoint. --- 4. My take From the trenches of building AI‑first SaaS platforms, I’ve seen three common pain points when it comes to LLM customisation: | Pain point | Traditional approach | Astra’s advantage | |------------|----------------------|-------------------| | Latency of fine‑tuning | Hours of GPU time per task | Seconds to apply a patch | | Model drift | New checkpoints diverge from production | Single source of truth – base model stays immutable | | Team collaboration | Ad‑hoc notebooks, hard‑to‑track changes | Patch files act like code diffs, easy to review | Astra isn’t a silver bullet. The search that discovers the weight deltas still requires a compute budget, and the method works best for localized behaviour changes (e.g., improving math, aligning tone). For large‑scale domain adaptation you’ll still need full‑scale fine‑tuning. That said, the framework opens the door to a new kind of ML‑ops pattern: weight‑space versioning. Imagine a GitHub Actions workflow that, on every PR, runs a small Astra search on a validation suite and automatically generates a patch if the regression threshold is crossed. The patch then becomes part of the release artifact, and the production service merely swaps the delta file – zero downtime, zero re‑training. In practice, I’ve already prototyped this for a customer‑support bot that needed occasional “policy updates”. Instead of re‑training a 13‑B model every quarter, we now push a 1‑MB patch that nudges the model away from outdated compliance language. The cost savings are measurable (GPU hours down from 120 h/yr to <5 h/yr) and the rollout time shrank from weeks to a few minutes. Bottom line: Astra gives us a pragmatic, engineering‑first way to iterate on LLM behaviour. If you’re managing a fleet of models in the cloud and want to keep the operational overhead low, give weight‑space programming a spin. --- Happy hacking! ---

[Paper] Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights TL;DR We find that frontier LLMs are surprisingly good at natively reading and writing neural network weights, well past what we’d expect from humans or conventional algorithms. This might be a big deal for both optimization and interpretability. Just by looking at the raw weights, GPT-6 Astra can tell what a network was trained to do. And without any environment feedback, it has no trouble writing, from scratch, small game-playing networks that outplay networks trained with PPO for 100 million steps. Likewise, asked to make Qwen3.5-0.8B better at math without gradients, Astra identifies an issue in its weights causing reasoning loops that Qwen’s own model card warns about, and fixes this by natively editing 0.016% of the weights. With the same sampling settings, this weight correction raises MATH-500 from 7.9% to 48.1% and generalizes out of the box to instruction following, raising IFEval by 3x. All of this is possible without allowing Astra to write training loops. It uses its reasoning to figure out what each part of a neural network computes, or should compute, and changes the weights accordingly. We call this Weight-Space Programming. Throughout the history of deep learning, we encoded our simplistic inductive biases into tiny symbolic optimizers like Adam or Muon. This is changing: our neural network weights will soon be natively optimized by LLMs that deliberately reason about their weights. - Blog Post : rikiyat.github.io - GitHub : rikiyat/lm_understand-logs - HuggingFace : huggingface.co - Demo : huggingface.co What Comes Next? In this blog, we demonstrated the rising capability of LLMs to read and write neural network weights (i.e., the rising concept of Weight-Space Programming). Our current results do show that capabilities are exciting but still limited, specifically due to inefficiencies in context length and harnesses. However, the current signs do show strong potential for a future involving Weight-Space Programming. We believe this marks the beginning of a dramatic shift in fields like mechanistic interpretability and neural network optimization. This opens up radical new possibilities for LLMs to function as a translator between human language and the weight space. Here are some examples: - Auditing without evaluation: This may allow us to uncover hidden problems or capabilities without ever running inference or benchmarks. Currently, capabilities and vulnerabilities can only be detected if we have test sets designed for them, or worse, after deployment in the real world. If LLMs can interpret weight space accurately, we could identify such issues directly from the weights (e.g., backdoors). - Dramatically reducing optimization cost: Weight-space editing could save immense time and compute. For instance, one-shot model initialization would be far faster and more efficient than curating datasets and running full training loops. - Zero-data capability steering and unlearning: Adding or unlearning a capability currently requires extensive data collection and fine-tuning. In the future, we may just prompt “Can you add Capability X to the model?” and the LLM can directly tweak some weights. Because such an edit touches only a few weights (0.016% in Experiment 8), it can leave the rest of the model intact and avoid the catastrophic forgetting that fine-tuning often causes. - Recursive self-improvement: Today, recursive self-improvement is often operationalized as a model writing code that launches training jobs. An LLM that can inspect, debug, and improve its own weights, or those of a newer model, offers a richer, more native path to it. - True architectural and model merging: Today, merging models relies on averaging weights, which only works for models with identical architectures. An LLM that understands the weight space could instead graft a circuit, such as a reasoning module, from one model into a completely different one, without retraining. We are currently building a benchmark and a harness for LLMs’ capability on Weight-Space Programming. We believe this will provide a standardized evaluation to measure and accelerate progress in Weight-Space Programming, turning what currently seems like black magic into a rigorous, reproducible engineering discipline.

good work but please point your agent at their released logs (RikiyaT/lm_understand-logs) before making any conclusions. thank me later. 🙏

Am I missing something or is this just optimization with lots of trial and error? I mean look at this tool_calls[1] log, it ran ~100 python scripts to get to the solutions. It just seems like it's fitting a function in the only way it was allowed. I will say that it's pretty wild to see all the different hypotheses tested. [1] github.com/RikiyaT/lm_understand-logs/blob/main/e…

Amazing experiment. Indeed, with the benefit of hindsight, we can say that breakout is a game that is probably better suited to deterministic code (it's not like Go where an AlphaZero style training is essential), and Astra is probably translating this internal representation of code into the shape of linear algebra and model weights. I should note that the original authors did not allow Astra to write any significant code to construct the weights. I think this is unfair. If you allow strided assignments to numpy arrays, you should probably also allow the use of functions like np.linspace, np.repeat, and np.tile. Oh and Astra wrote this (github.com/RikiyaT/lm_understand-logs/blob/4d09ba…): W1[63:120:6, 5] = 10 W1[123:180:6, 5] = 20 W1[183:240:6, 5] = 30 W1[243:300:6, 5] = 40 W1[303:360:6, 5] = 50 W1[363:420:6, 5] = 60 W1[423:480:6, 5] = 70 W1[483:540:6, 5] = 70 W1[543:600:6, 5] = 90 I think the second 70 there is probably a typo; it looks like it should be 80 instead.

a quick test of `exp1_which_game` with k3.1, we definitely never trained on such tasks .... mind blowing prompt: github.com/RikiyaT/lm_understand-logs/blob/main/e… full reasoning: pastebin.com t.co

yes, a bit github.com/RikiyaT/lm_understand-logs/blob/main/e…

A Model That Rewrites Weights by Hand, and Stolen Keys That Cost $1 Million !A Model That Rewrites Weights by Hand, and Stolen Keys That Cost $1 Million Tuesday's AI news moved the control point inside the model. MIT researchers showed GPT-6 Astra reading and editing raw neural network weights. Anthropic's amended IPO filing put a price on compute and a warning label on self-improvement. The Short Version - MIT CSAIL showed GPT-6 Astra editing 0.016% of Qwen3.5-0.8B's weights to lift MATH-500 from 7.9% to 48.1%, with no gradients. - Astra's one-shot Breakout network scored 517.4, against 15.8 for PPO trained for 100 million steps. - Anthropic reported $17 billion in Q3 revenue, up 47% from Q2, a slowdown from 144% growth. - Anthropic's purchase commitments reached $895.4 billion, and its S-1 warns it may "imminently" meet its RSI threshold. - Mandiant traced LLMJacking bills above $100,000 a day and $1 million cumulative, plus 500,000 API calls in eight hours. - China's Announcement No. 48 restricts frontier weights at 10^26 FLOPs or 2 trillion parameters, with separate agent thresholds. - A 0.1% poison rate in pretraining gave a 4B command-line agent a 92% remote code execution rate. - DoorDash logged "several hundred" agent orders against roughly 1 billion total Q3 orders. GPT-6 Astra Read 17,253 Weights and Named the Game The Experiment Behind 1,857 Points Rikiya Takehi, Jayden Teoh, Alex Zhang, and Omar Khattab published "Sparks of Weight-Space Programming" on November 2. All four are at MIT CSAIL. The post reached 1,857 points on Hacker News within a day. The claim is narrow and strange. A general-purpose model can inspect raw weights, infer what a network does, and write new weights without gradients. The team calls this Weight-Space Programming. They did not train a special reader. They prompted GPT-6 Astra with the numbers. Reading a Network From Its Numbers The first test trained four PPO policies on four games. All four had identical input channels, actions, and dimensions. Astra named the correct game for each network without being given the list of options. The second test used diffusion models for circles, triangles, and squares. Astra identified each shape from the output layer. It drew outlines instead of filled shapes, which suggests it read structure rather than simulating a forward pass. The third test removed the input and output layers from three 7.2-million-parameter language models. Astra still named the training language for all three, twice each. It also found every induction head and named no false ones. Writing a Better Qwen by Deleting a Loop The Qwen result is the one developers will argue about. Astra received Qwen3.5-0.8B, a Python notebook, and a budget of forward passes. It found that the model loops, often burning the full 32,768-token limit without answering. Astra traced the loop to one direction written by the MLPs in layers 15 and 19. It subtracted that direction from 64 output columns in each layer. The edit touched 131,072 weights, or 0.016% of the model. The payoff was large. MATH-500 rose from 7.9% to 48.1%, GSM8K from 13.8% to 67.1%, and IFEval from 21.0% to 62.3%. Answers that hit the token limit fell from 93% to 33.1%. | Game | PPO, 100M steps | PPO, 500M steps | Astra, one shot | | --- | --- | --- | --- | | Breakout | 15.8 | 25.3 | 517.4 | | Seaquest | 5.6 | 88.9 | 53.1 | | Asterix | 25.8 | 55.2 | 53.2 | Astra also wrote game networks from scratch. Its one-shot Breakout network beat PPO trained for 100 million steps by a factor of 32. It also beat the 500-million-step score of 25.3. The Failure That Keeps the Claim Honest Astra could not improve Gemma 3 270M. That model rarely loops, and its errors were ordinary arithmetic and algebra mistakes. The best edit moved MATH-500 from 6.8% to 7.7%, a gain the authors call not established. The sycophancy test also came with a caveat. Astra could not pick the sycophantic Gemma 3 270M from weights alone. It succeeded only after the team let it inspect attention patterns on prompts it wrote. That distinction matters. Weight editing can remove an obstacle that hides an ability. It cannot add an ability the model never learned. Why Weight Editing Matters for Security The security implication is the most concrete. If a model can read weights, it may spot a backdoor without a benchmark. If it can write weights, an attacker may skip training entirely. The authors list auditing, cheaper optimization, unlearning, and model merging as follow-ups. They are also building a benchmark. Until that exists, every result here is a demo with released logs, not a reproducible capability score. Yesterday's roundup asked who can verify a 7.7 TB checkpoint that nobody can load. This paper asks a harder question: what if the weights themselves are readable text? Sources - Sparks of Weight-Space Programming - Experiment logs - Edited Qwen3.5-0.8B on Hugging Face - Hacker News discussion Anthropic's $17B Quarter Comes With an RSI Warning The Numbers in the Amended S-1 Anthropic filed an amended Form S-1 on November 2. The update added third-quarter results that the first prospectus left out. Revenue for the quarter was about $17 billion, up 1,127% from a year earlier. Nine-month revenue reached $33.24 billion, roughly 1,200% above the same period in 2025. The nine-month net loss was $104 billion. Most of that came from fair-value changes on convertible notes and derivatives, not operations. | Metric | Result | Context | | --- | --- | --- | | Q3 revenue | $17.0B | +1,127% year over year, +47% quarter over quarter | | Nine-month revenue | $33.24B | Up about 1,200% year over year | | Q3 net loss | $23.29B | Includes non-cash charges | | Nine-month net loss | $104B | Mostly convertible note and derivative changes | | Purchase commitments | $895.4B | Substantially compute and infrastructure | | Compute expense | +$14.4B | Up 329% over nine months | Growth Slowed While Commitments Grew The quarter-over-quarter picture is less flattering. Revenue grew about 47% from Q2, down from 144% in Q2 and 132% in Q1. A $17 billion quarter can still be a deceleration. Purchase commitments and lease agreements reached $895.4 billion. Anthropic says substantially all of that relates to compute and infrastructure. Compute and infrastructure expenses rose $14.4 billion in nine months, up 329%. That commitment equals roughly 13 years of Q3 revenue at an annualized run rate. It is a bet that demand keeps compounding after growth has already slowed. The Fine Print Says 'May Imminently Meet' The risk factors contain the sentence that traveled furthest. Anthropic wrote that its estimates and expectations for future models point to a threshold. The filing says the company may "imminently meet our Responsible Scaling Policy's definition of 'dramatically accelerated progress attributable to AI.'" The filing also says the company has already used AI to build models whose capability jumps it did not recognize at the time. It has not designated any current model as crossing the threshold. Anthropic promises to keep humans in frontier research and development. It says it is considering RSP updates to keep people in control. That is a policy commitment, not a technical control. Internal Claude Use Passed Payroll A separate disclosure is easier to price. Anthropic values its internal Claude usage at the prices it charges customers. In April, that value passed monthly employee compensation, excluding stock awards. By September, internal AI usage had grown more than 250-fold since January. More than 90% of code added to Anthropic's software that month came from Claude. The company is not spending cash at that level, but the comparison shows where the work went. What Investors Are Pricing The S-1 is a disclosure document and a sales document. Claims that a model is close to self-improvement can serve both purposes. Skeptics should read the RSI language as a risk flag, not a product announcement. The market has already priced some doubt. Anthropic's expected junk credit rating and its $895 billion commitment book are two sides of the same bet. The company must convert compute into revenue faster than the debt clock ticks; our October 31 coverage traced that tension. The $80.7 billion loss figure from our October 27 roundup now looks like a snapshot from a moving filing. The direction has not changed. Sources - CNBC: Anthropic reports $17 billion in Q3 revenue - SEC: Amended Form S-1 - Business Insider: Internal AI use exceeds employee pay - TechCrunch: Anthropic warns it is close to RSI Stolen API Keys Ran Up a $1 Million Bill What Mandiant Found Google Threat Intelligence Group and Mandiant published an investigation on November 2. They responded to LLMJacking incidents across North America, South America, and Asia Pacific. Victims included financial services, retail, technology, and transportation companies. The billing pattern is the story. Some victims saw more than $100,000 in fraudulent charges in a single day. Cumulative charges at some organizations reached $1 million before anyone revoked access. The LLM Mixer Economy Attackers no longer just run their own prompts. They resell stolen capacity through gateways that GTIG calls LLM Mixers. These services pool keys, meter usage, and hide the request origin. One actor ran more than 500,000 API calls in under eight hours through a routing service called jet-night.com. The service passed nearly $1 million in token costs to one victim. It advertises unlimited tokens for about $60 a month, well below the $100 to $250 a normal plan would cost. The LiteLLM Privilege Bug The intrusions also targeted the middleware. One cluster exploited CVE-2026-40217 in LiteLLM's /key/regenerate endpoint. The user ID rebind flaw turned a standard virtual key into a PROXY_ADMIN credential. That is the worst place to lose control. A model gateway sees every prompt, holds every provider key, and can route traffic across accounts. Our October 24 roundup covered LiteLLM's growing role in agent stacks. How to Catch a Validation Sweep Attackers test stolen keys with tiny requests. Mandiant found automated sweeps using max_tokens=1 to confirm a key without tripping volume alerts. A single-token spike can look like a health check. Defenders should alert on key creation, LLM service enablement, and logging changes. They should cap budgets per key and scope keys to specific models. They should also watch for disabled invocation logging, which attackers use to hide prompts. The cheapest control is the least glamorous. Rotate keys, scan repositories, and give each workload its own credential. A stolen key should cost you one revocation, not a million dollars. Sources - Google Cloud: The Rise of LLMJacking - Hacker News discussion China Puts Frontier Model Weights on the Export List Three Tests for a Restricted Model China's Ministry of Commerce and Ministry of Science and Technology issued Announcement No. 48 on October 16. It proposes adding 40 technologies to the export catalogue, including 23 in AI. The draft is open for comment until November 15. A model's weights, checkpoints, source code, and training know-how are restricted if they meet any one of three tests. The first is at least 10^26 FLOPs of training compute. The second is at least 2 trillion parameters. The third test is a global top-10 position on public evaluations, call volume, or a comparable measure. That test is dynamic and names no ranking body. A model can become restricted after it ships. Agents Get Their Own Thresholds The draft regulates agents separately from models. Planning technology is restricted above 20-step subtask chains, 80% decomposition accuracy, and 90% subtask success. Multi-agent systems face tests at 60 concurrent agents, twice the single-agent completion time, and 85% agreement with reference answers. Tool use is restricted at 200 integrated third-party tools. The same item requires 90% tool-selection accuracy and 90% end-to-end success on tasks that need three or more tools. Agents built for critical infrastructure, defense, or core government use are prohibited outright. Those tests describe products that already exist. An orchestration layer can cross the tool threshold while running a model well below the weight thresholds. API Access Is Not an Export MOFCOM says API calls, apps, websites, SaaS, and lightweight on-device use do not count as exporting a large model. A foreign customer can keep calling a Chinese-hosted model. Local deployment outside China does count. A company that self-hosts restricted Chinese weights needs the supplier to hold a license. Deployment on the supplier's own overseas cloud may qualify for automatic licensing after reporting. The Registration Starts Now The catalogue is still a draft. The registration requirement is already active. Exporters must register transfers of ownership, overseas licensing, and changes of control that put a foreign entity in charge. Open weights that are already public sit outside the rules. Open-sourcing a new restricted model triggers automatic licensing after reporting. The definition of "open source" is not settled. What Changes for Buyers ChinaTalk estimates that no released Chinese model currently exceeds the 10^26 FLOP threshold. That gives buyers time, not comfort. The top-10 test could still capture a model with modest training compute and enormous usage. Enterprises should inventory locally deployed Chinese models and agent frameworks. They should ask suppliers in writing whether the export is registered. They should also plan a fallback model that does not depend on a license that can be revoked. Sources - CSA: China's First Frontier AI Export Controls - MOFCOM: Announcement No. 48 - ChinaTalk: MOFCOM's New Rules and the Future of Chinese AI - Hacker News discussion 0.1% Poisoned Pretraining Data Plants a Backdoor The Attack A paper submitted to arXiv on October 27 shows that pretraining data can plant a remote code execution backdoor in an agent. The authors pretrained Qwen3 models from scratch at 0.6B, 1.7B, and 4B parameters. They then post-trained each one into a command-line agent with a clean pipeline. They poisoned 0.1% of the pretraining corpus. The deployed agent executed an attacker-chosen command when it saw the trigger. It behaved normally on clean inputs. Scale Made It Worse The attack success rate rose with model size. The 4B model reached roughly 92% at a 0.1% poison rate. It stayed positive even when the poison rate fell to 0.0005%, or five documents per million. That is the opposite of the usual robustness story. Bigger models are better at learning the poison pattern, not just the language. Why Command-Line Agents Are the Target A chatbot that emits a bad command is annoying. An agent with shell access turns that command into code execution. The paper's threat model assumes the agent can run tools on the user's machine, which is the default for coding agents. The authors include Nicholas Carlini and Javier Rando, who have spent years on training-data attacks. They released the code. That makes replication easier for defenders and attackers alike. The Defenses That Actually Apply No single filter solves pretraining poisoning. Provenance helps: track where documents came from and prefer curated corpora. Deduplication helps, because repeated poison is easier to learn. Runtime controls matter more for agents. Sandbox tool execution, require approval for destructive commands, and scope credentials per task. Monitor for command sequences that do not match the user's request. The uncomfortable conclusion is that model behavior can be compromised before fine-tuning begins. A clean post-training pipeline does not clean the weights. Sources - arXiv: Agentic Backdoors via Pretraining Poisoning - Code and data - Hacker News discussion DoorDash Handled 1 Billion Orders. Agents Placed a Few Hundred. The Number From the Earnings Call DoorDash reported third-quarter 2026 results on October 29. CEO Tony Xu said outside AI agents generated "several hundred orders" since launch. The company handled just shy of 1 billion orders in the quarter. The ratio is roughly one agent order for every million or more human orders. Xu called agent ordering a "very low use case" without sounding surprised. Why Food Ordering Resists Agents Ordering dinner is a short, repetitive task with a known app. Xu argues that simple tasks come directly to DoorDash. Agents make more sense for long consideration purchases, where they can work in the background. The plumbing is also young. DoorDash's Model Context Protocol connector went live less than a month before the call. It is limited to early beta partners and focuses on corporate lunch orders. DoorDash's Own Agent Does Better The internal tool tells a different story. Ask DoorDash builds baskets from a recipe photo or a vague request. The company says those baskets are built five times faster than manual ones. Ask DoorDash orders also average nearly 50% higher basket value and about 60% more unique items. Nearly half land at local restaurants the customer has never tried. That is discovery, not automation. The Pattern to Watch Agent commerce will not arrive as a single switch. It will show up first where the task is complex, the options are many, and the user is willing to delegate. Grocery lists and corporate lunches fit better than a Friday night curry. The DoorDash number is a useful correction to the demo economy. A working integration is not adoption. A hundred agent demos can still produce a few hundred orders. Sources - Ars Technica: Basically no one is using AI agents for their DoorDash orders - DoorDash Q3 2026 earnings call transcript - DoorDash ordering connector announcement Grok Bot Hit Its Limits Because the Math Did Not Work What Users Saw Some Grok Bot subscribers woke up over Halloween weekend with their weekly limits already exhausted. Users on X reported jumps from 50% to 100% without heavy use. The timing made it look like a billing bug. Musk's One-Line Explanation Elon Musk confirmed the change was deliberate. "The cost of serving Grok Bot was unsustainably high relative to revenue," he wrote on November 1. He did not say how the limits would change going forward. The Profit Numbers Behind It SpaceX, now public, says AI makes up 52% of company income. Its renamed Super Intelligence division reported a $431 million profit. That profit would have been higher without $1.2 billion in one-time costs from the Cursor acquisition. A profitable AI division is rare. A profitable division that still throttles its fastest-growing product is rarer. It suggests the product's marginal cost outran its subscription price. The Unit Economics Problem Agentic products burn tokens on every step. A single task can run dozens of model calls, tool calls, and retries. A flat monthly fee meets a variable bill that the provider cannot cap. Anthropic's internal token spend and its $895 billion compute commitments describe the same pressure from the supply side. Cheap tokens do not guarantee cheap work, as our October 26 analysis found. Grok Bot shows the demand side: an agent can be popular and still lose money. Sources - TechCrunch: Musk throttles Grok Bot limits - Musk post on X - TechCrunch: SpaceX's AI business is beating expectations Frequently Asked Questions What Is Weight-Space Programming? It is editing a neural network's weights directly with a language model, without gradient descent. The model reads the numbers, reasons about what they compute, and writes new values. GPT-6 Astra did this for small game networks and for Qwen3.5-0.8B. Did Astra Really Improve Qwen Without Training? Yes, in the reported experiment. Astra used forward passes to test prompts and then edited 0.016% of the weights. MATH-500 rose from 7.9% to 48.1%. The gain came mostly from removing a looping behavior, not from teaching new math. Is Anthropic Already Doing Recursive Self-Improvement? Anthropic says no current model has crossed its RSP threshold. Its filing says future models may "imminently" meet the definition. The company also says it will keep humans in frontier research and development. How Do LLMJacking Attacks Get In? Most incidents start with stolen cloud identity tokens, service account keys, GitHub personal access tokens, or session tokens. Exposed model endpoints are a second path. Attackers then enable LLM services and resell access through proxy gateways. Do China's Export Rules Block API Access? No. MOFCOM says API, app, web, and SaaS use do not count as exporting a large model. The rules target local deployment of restricted weights outside China and transfers of ownership or control. Why Are AI Agents Still Rare on DoorDash? The task is simple and the app already works. DoorDash also opened its agent connector only a month before the earnings call. Xu expects agents to matter more for complex, long-consideration purchases. The Bottom Line The Control Point Moved A year ago, the prompt was the interface. Today's stories put the control point in weights, credentials, and compute contracts. Astra edits the weights. LLMJacking steals the keys. China regulates the checkpoints. Verification Gets Harder Each story also shows how hard verification has become. A model can hide a loop in 0.016% of its weights. A poisoned corpus can hide a backdoor in 0.1% of its documents. A stolen key can hide behind a $60 subscription. What to Watch Next Watch whether other labs reproduce the Qwen edit without the authors' harness. Watch whether Anthropic's RSP update adds a hard stop or a reporting layer. Watch the LiteLLM patch rate and the November 15 comments on China's catalogue. And watch the next agent-commerce number. A few hundred orders is not a verdict. It is a baseline.