I ported autoresearch to Claude Code: it edits, benchmarks, keeps what's faster and reverts the rest, on a loop
What it is
pi-autoresearch (by Tobi Lutke & David Cortés, inspired by karpathy/autoresearch) runs optimization loops on its own inside the pi agent. I ported it to Claude Code as a plugin. MIT, free.
You give it a goal and it loops: change code → run the benchmark → commit if better, git-revert if worse → repeat.
/plugin install autoresearch --marketplace victor-falcon/claude-autoresearch
/autoresearch optimize unit test runtime, monitor correctness
It works for anything you can measure with a command: test time, bundle size, build time, training loss, Lighthouse. An optional checks.sh (tests/lint) has to pass before a change is kept, a confidence score (best improvement vs. the session's noise) keeps jitter from counting as a win, and /autoresearch finalize splits the experiment branch into clean, independent PR branches.
How it's built
It's a mod (function-hooks plugin) plus three plugin skills. I split the port into seven tasks, each merged as its own PR: a pure core library ported from upstream with unit tests and CI first, then the three experiment tools, the skills, the terminal UI (widget above the prompt, status line, fullscreen dashboard), the /autoresearch command with the auto-resume loop, the browser dashboard, and finally docs and end-to-end QA. The .auto/ files and the JSONL log keep upstream's format, so a session can move between pi and Claude Code.
What I learned
- Long unattended loops need their state outside the conversation. Everything lives in .auto/log.jsonl and .auto/prompt.md, and after a compaction the plugin rebuilds the context from those files instead of trusting the summary.
- Auto-resume needs brakes: it stops after 20 consecutive discards or crashes, at maxIterations, and never re-prompts after Escape, a refusal or an API error.
- Org managed settings can block a user plugin's system-prompt and compaction hooks, so the plugin detects that and sends the same guidance with the prompts instead.
- Claude Code reserves plugin names starting with claude-, so the plugin is just autoresearch.
- In 0.1.0 the mode turned itself on when you opened Claude Code in a folder with a session log, which undid /autoresearch off. 0.2.0 only turns it on when you ask.
Heads-up: run it on a clean branch, it commits and reverts by itself. Needs Claude Code 2.1.300+.
Repo: victor-falcon/claude-autoresearch
1 point · 4 comments
Mentions
Highly recommend this Pi plugin davebcn87/pi-autoresearch If you have the right kind of problem this can be a great way to approach it.
I ported autoresearch to Claude Code: it edits, benchmarks, keeps what's faster and reverts the rest, on a loop What it is pi-autoresearch (by Tobi Lutke & David Cortés, inspired by karpathy/autoresearch) runs optimization loops on its own inside the pi agent. I ported it to Claude Code as a plugin. MIT, free. You give it a goal and it loops: change code → run the benchmark → commit if better, git-revert if worse → repeat. /plugin install autoresearch --marketplace victor-falcon/claude-autoresearch /autoresearch optimize unit test runtime, monitor correctness It works for anything you can measure with a command: test time, bundle size, build time, training loss, Lighthouse. An optional checks.sh (tests/lint) has to pass before a change is kept, a confidence score (best improvement vs. the session's noise) keeps jitter from counting as a win, and /autoresearch finalize splits the experiment branch into clean, independent PR branches. How it's built It's a mod (function-hooks plugin) plus three plugin skills. I split the port into seven tasks, each merged as its own PR: a pure core library ported from upstream with unit tests and CI first, then the three experiment tools, the skills, the terminal UI (widget above the prompt, status line, fullscreen dashboard), the /autoresearch command with the auto-resume loop, the browser dashboard, and finally docs and end-to-end QA. The .auto/ files and the JSONL log keep upstream's format, so a session can move between pi and Claude Code. What I learned - Long unattended loops need their state outside the conversation. Everything lives in .auto/log.jsonl and .auto/prompt.md, and after a compaction the plugin rebuilds the context from those files instead of trusting the summary. - Auto-resume needs brakes: it stops after 20 consecutive discards or crashes, at maxIterations, and never re-prompts after Escape, a refusal or an API error. - Org managed settings can block a user plugin's system-prompt and compaction hooks, so the plugin detects that and sends the same guidance with the prompts instead. - Claude Code reserves plugin names starting with claude-, so the plugin is just autoresearch. - In 0.1.0 the mode turned itself on when you opened Claude Code in a folder with a session log, which undid /autoresearch off. 0.2.0 only turns it on when you ask. Heads-up: run it on a clean branch, it commits and reverts by itself. Needs Claude Code 2.1.300+. Repo: victor-falcon/claude-autoresearch
@mattpocockuk it is already thoughtfully implemented here davebcn87/pi-autoresearch