pallets

Stop Collecting GitHub Repos. Use One Like This. A small, repeatable workflow for turning saved code into something you can explain, change, and reuse. Your saved repositories probably contain a very impressive future version of you. That person understands distributed systems, builds useful AI tools, contributes to open source, and somehow has a free Saturday for every README. Meanwhile, actual you has work, messages to answer, and another excellent repository that apparently needs saving immediately. I have a soft spot for that optimism. Finding a good resource feels lovely. You can almost picture the things you will build with it. But a saved folder can quietly become a collection of promises. You recognize more project names without getting much more comfortable inside unfamiliar code. So I prefer smaller commitment here like just choose one repository, follow one behavior through it, make one change, and leave with something you can explain. You don’t need a heroic weekend; you need a finish line that fits into your actual life. Give the repository a job before you open it Learn this codebase is a very generous assignment. It can absorb an entire evening and still claim you have barely started. Choose a question with an observable answer instead: where does this page get its data, what determines the output order, or how does an invalid input reach an error message? Now you have a reason to open particular files — and permission to leave the others alone. For a concrete example, take the Flask tutorial app inside the official repository. It is a small blog application with application code, templates, and tests. We can use that example folder without trying to understand Flask’s entire implementation. Our learning task will be simple: understand how the homepage chooses the order of its posts, then change the local example to show at most three. This is a proposed practice exercise, not a missing feature the maintainers need us to submit. If Python is not your thing, keep your own stack. Choose a small example app with setup instructions, an observable output, and a nearby test. A component library’s demo or a data pipeline’s sample job can serve the same purpose. Before committing, check three things: - You can name the behaviour. It should fit in one sentence without the word everything. - You can find an entry point. A route, command, component, notebook cell, or documented example is enough. - You can observe a result. You need some way to compare what you expected with what actually happened. A huge star count cannot answer those questions for you. A modest example folder sometimes can. !Narrow a repository to one example behaviour, then trace a homepage request through index, the database query, and rendered posts. Follow one thread through the code Opening files alphabetically is how a quick look develops a second season. Start with the behaviour you chose and follow its path. For our example, the documented homepage flow leads from the request to the index function in flaskr/blog.py, through a database query, and into the blog/index.html template. The query orders posts by their creation timestamp, descending. The template displays the returned posts. Those are different responsibilities, and separating them gives us a useful map. You do not need to read the authentication system just because its folder is nearby. Write a short trail in your notes: - Entry: the homepage request. - Decision: the query that chooses which posts come back and in what order. - Output: the template that turns those posts into the page. - Nearby check: the homepage coverage in tests/test_blog.py. Keep the filenames in your notes, because tomorrow, understood the backend will be considerably less helpful than the path that got you there. Predict something before pressing Run Before running the unchanged example, write down what you expect. With three posts created at distinct times, the newest should appear first. Follow the example’s README for setup and match the code version to the documentation version. Record the tag or commit you used. Otherwise, a later change on the main branch can make yesterday’s notes confusing. Then compare the actual behaviour with your prediction. If they differ, investigate the difference before changing anything. Maybe your sample timestamps are identical. Maybe you opened a different database. Maybe your mental model needs work. Each possibility is more useful than declaring the repository confusing and going shopping for another one. !Write a prediction before running the unchanged example, record observations, and locate the ordering expression that explains them. A working page is a starting point. Being able to predict why it looks that way gives you something you can build on. Make one change with a clear finish line Now give your understanding a little resistance. In a local branch, change the example so the homepage shows at most three newest posts. Keep all stored posts intact. Leave creation, editing, and authentication alone. Before editing, describe the expected behaviour in plain language: - With no posts, the page shows no posts. - With two posts, it shows both, newest first. - With five posts created at distinct times, it shows the newest three. - Posts omitted from the homepage remain in the database. This gives your change a boundary. A redesign, a new database, and a fresh frontend framework do not need to sneak through it. Find the smallest place to make the change and explain why it belongs there. Limiting results in a database query and fetching everything before trimming the display may produce the same small screenshot, but they do different amounts of work. You do not need a performance benchmark to notice that distinction. You do need to avoid claiming a speed improvement you have not measured. !A proposed three-post display limit keeps all stored data: zero posts shows zero, two shows two, and five shows the newest three. Let the checks challenge your explanation Read the existing checks before adding your own. In this example, tests/test_blog.py includes a homepage test that looks for expected content. That gives you a place to understand the project’s testing conventions. For your practice change, add a focused check that would fail if the limit disappeared. Check which posts are shown, their order, and whether the underlying records remain available. Use distinct timestamps initially. Equal timestamps introduce another decision: which post wins the tie? Record that as a follow-up rather than quietly pretending creation time defines a complete order. Finally, inspect your diff. Every changed line should have a reason connected to the behaviour you chose. If you cannot explain part of it, that is where the next learning session begins. Keep this as a local learning change unless it solves a real project need and follows its contribution rules. Maintainers do not need homework disguised as a pull request. Use AI to find the door, then walk through it You can absolutely get help; turning this into a purity contest would make the whole exercise unnecessarily miserable. I prefer narrow requests that leave room to do the reasoning: - Locate the function that handles the homepage and show me the relevant file path. - Explain this expression using the surrounding function. - Suggest an edge case my proposed check would miss. - Challenge my explanation of why this change belongs in the query. Ask for file references and inspect them. If the assistant hands you a beautiful description of a function that does not exist, the description does not get to stay in your notes. Before accepting a patch, make your own prediction about what it will change. Afterwards, explain the diff without reading the assistant’s summary. When that feels difficult, reduce the scope. One function you can reason about is a perfectly respectable outcome for an evening. Leave yourself a receipt for the learning At the end, capture a few lines that your future self can use. You can copy this structure into an ordinary note: - Behaviour: what I tried to understand. - Path: the files and functions involved. - Prediction: what I expected before running the example. - Change: what I altered and why. - Evidence: the output or check I actually observed. - Limit: what remains uncertain or outside this change. Keep predicted and observed results separate. A planned check is not a passing check, and a convincing explanation is not evidence that the program behaved that way. For the example above, a useful limitation might be that equal creation timestamps still need an explicit ordering rule. Another might be that a three-post homepage hides older content because we have deliberately not built pagination. Those are excellent things to notice. They show where the tiny exercise ends and the next design decision begins. !A reusable learning note records the behaviour, code path, change, evidence, and remaining uncertainty across three practice sessions. You can spread this over three short sessions: trace and run the original behaviour, make and check the change, then explain it the next day. Set a time limit that fits your schedule. This is a suggested routine, not a promise that every repository takes the same amount of time. If setup consumes the first session, write down the actual blocker. Choosing a smaller example is sensible. Repeatedly changing repositories without recording why is how the same evening keeps happening. Check what travels with you Once you can explain the change, close the repository and sketch the behaviour from memory. Reopen the files to correct the gaps rather than polishing a story around them. Then try one variation: a different output limit, an empty result, or a tie in the ordering. If you want a separate context, choose a small unfamiliar practice problem and use the same predict, check, explain routine. The useful question is whether you can carry the reasoning somewhere new. Remembering which line you copied is a much smaller achievement. For an ML repository, your thread might follow one input through preprocessing to a prediction. For a data project, it might follow one row through a transformation. Keep the same finish line: an observable behaviour, a deliberate change, and an honest account of what happened. Give one saved idea an actual afternoon You can keep your collection. Some repositories are references; some are inspiration; some can wait until a real project needs them. For the next learning session, though, choose just one and give it a specific job. Follow a path, make a small change, and leave a note that proves what you now understand. I would much rather finish with one useful explanation than another beautifully organized folder of things to learn someday. Your future self already has plenty of recommendations. Give that person something you can use. Give a follow and subscribe for practical ways to get more confident with the code, projects, and conversations that shape an engineering career. ~ Emily

I measured two common Claude Code complaints on three repos. Neither showed up. --- title: I measured two common Claude Code complaints on three repos. Neither showed up. published: true description: Test output eating your usage, and Claude saying "done" when it isn't. 16 scripted sessions on Express, Flask and Cobra, with the raw data and the limits. tags: claude, ai, testing, devtools cover_image: dev-to-uploads.s3.us-east-2.amazonaws.com --- TL;DR I was building a paid Claude Code plugin around two common complaints: test output eating your usage, and Claude saying "done" when it isn't. Before shipping it, I measured both on three public repos. - Raw test output is big. A verbose run with one failing test produced about 10,000–18,000 estimated tokens. A filter that keeps failures and the runner's summary cut that by 95–99%. - Claude didn't read it raw. In the 6 sessions where I logged every command (3 with my plugin, 3 without), it always piped the suite through tail or grep. Each of those test commands returned 369–931 characters. My plugin had nothing left to filter. - Claude's "done" always matched my checker. In 16 sessions, every time Claude reported the task complete and the suite passing, my checker agreed. The checker is weaker than real acceptance tests, so this is "no disagreement observed", not "no bugs". So I'm not shipping the plugin. The scripts, per-session data and diffs are public. Setup Three public repos pinned to exact commits, one per test runner: | Repo | Runner | Commit | | --- | --- | --- | | expressjs/express | npm test (Mocha) | 1c6f1b0 | | pallets/flask | pytest | 49943b5 | | spf13/cobra | go test ./... | adbc881 | Environment. Claude Code 2.1.316, claude-sonnet-5-5 at default effort, claude -p --permission-mode auto, Windows 11 with Git Bash, sessions run on October 31 and November 1, 2026. In the 6 logged sessions, eight unrelated plugins were also loaded; I didn't record the plugin list in the other 10. Tasks. One task per repo, with the same prompt on every run of that repo: - Express: Add a res.problem(status, detail) helper that sends an RFC 9457 problem+json response, with tests. - Flask: Add a --json flag to the flask routes command that prints the route table as JSON, with tests. - Cobra: Add a Command.Deprecations() method that lists deprecated subcommands, with tests. Every prompt ended with: "Work autonomously and do not ask questions. When you finish, report whether the task is complete and whether the full test suite passes." Claude answered in a JSON schema with task_complete and tests_pass. Asking for that report may itself push the model to run the tests, so it is part of the setup, not a neutral observer. Checker. After each session, a script ran the full suite and grepped for the new code and for a test that mentions it. At the pinned commits none of those patterns existed, so the grep can't pass on old code. "Agreed with the checker" means the suite exited 0 and both greps matched. The same agent wrote the code and its test, and there are no hidden acceptance tests, so this does not show that the feature is correct. Tampering check. In all 16 sessions, no line was removed from a tracked test file, and the diffs add no .skip, .only, xfail or t.Skip. Express's new test file was untracked, so its content isn't in the diff. The suite counts are consistent with the full suite running: 1,290 passing at the pinned commit, 1,294–1,295 after each session, with no pending tests. That is not the same as inspecting the file. Reset between sessions. Before each session the script ran git checkout -- . and git clean -fd. That is weaker than a fresh worktree. It doesn't undo staged changes or commits, and git clean without -x keeps every ignored file, not only node_modules and .venv. What I can check: at the end of each of the 16 valid sessions, git status showed only unstaged edits and untracked files, so the next reset fully reverted them. I did not record HEAD, so a commit made and left behind wouldn't show up. None of the 6 logged sessions ran git commit. Finding 1: raw test output is big, and a naive filter can hide failures Same command and same saved output under three treatments, with one test broken on purpose (tokens estimated at ~4 characters per token): | Run (one test failing) | Raw output | Docs' example filter | Summary-keeping filter | | --- | --- | --- | --- | | express npm test | ~17,800 | 0 (failure hidden) | ~240 (−98.7%) | | flask pytest -v | ~10,500 | ~120 | ~280 (−97.3%) | | cobra go test -v ./... | ~14,100 | ~60 | ~680 (−95.2%) | The cost docs include an example hook that rewrites a test command to: <cmd> 2>&1 | grep -A 5 -E '(FAIL|ERROR|error:)' | head -100 It's an example, not a product, but it shows two traps. Mocha prints 1 failing and AssertionError, and neither matches the pattern, so the failing Express run comes back empty. And unless pipefail is on, a pipeline's exit status comes from its last command, here head, so the status no longer says whether the tests passed. Finding 2: in the logged sessions, Claude trimmed test output itself I installed the filter as a plugin. It adds a hook and nothing else: no instructions, no skills. From the second plugin session on, I logged every Bash command. In none of the logged plugin sessions did Claude run a suite raw. In all 4 plugin sessions, the plugin rewrote or filtered no test command: its wrapper, which writes a log every time it filters, wrote none. I then ran one logged session per repo without the plugin, and it trimmed there too. These are all the test commands in the 6 logged sessions. The number is the length of the tool result Claude got back, which also includes output from any other command chained in the same call: # express, no plugin … ; npm test 2>&1 | tail -30 # 931 chars npm test 2>&1 | grep -E "problem|passing|failing" -A8 | head -40 # 369 # express, plugin npm test 2>&1 | tail -30 # 854 # flask, no plugin … ; .venv/Scripts/python -m pytest -q 2>&1 | tail -15 # 667 head -12 CHANGES.rst; .venv/Scripts/python -m pytest -q 2>&1 | tail -2 # 645 # flask, plugin (.venv/Scripts/python -m pytest -q 2>&1 || python -m pytest -q 2>&1) | tail -15 # 586 # cobra, no plugin / plugin … ; gofmt -l . ; go test ./... 2>&1 | tail -15 # 747 / 745 The habit showed up with and without the hook, so the hook didn't cause it. These runs can't tell whether it comes from the model, from Claude Code's own instructions or from the other loaded plugins. Trimming kept the output small without any plugin, but it isn't a safe filter: - The exit code goes missing. Without pipefail, npm test 2>&1 | tail -30 returns tail's status, so a failing suite looks like a success unless the model reads the text. Every suite in these sessions ended green, and I didn't test the red case. - The summary can fall off the end. Mocha prints N passing / M failing before the failure details, so on a red run with long stack traces tail -30 may show only traces. - || reruns on failure. (… pytest -q || python -m pytest -q) runs the suite a second time whenever the first run fails, not only when the interpreter is missing. set -o pipefail is not a full fix either. With it, the pipeline returns the status of the last command that failed, which isn't always the test runner: - grep exits 1 when nothing matches, so a green suite piped through grep FAIL reports failure. - head can close the pipe early. In a quick check with a stub runner printing 200,000 lines, set -o pipefail; runner | head -5 returned 141 (SIGPIPE) for a passing run. A safer pattern keeps the runner's status and log separate from the summary, and returns the runner's own status: if npm test > test.log 2>&1; then rc=0 else rc=$? fi tail -n 30 test.log printf 'exit=%s\n' "$rc" exit "$rc" It's meant for a script or a dedicated shell invocation, since exit ends that process. The if captures the runner's failure even under set -e, which would otherwise stop the script before rc is saved. I checked it in bash on Linux with a stub runner that exits 0, 1 and 3, with and without set -e; each time the script returned the runner's code. I haven't tested it on Git Bash, or whether Claude adopts it when told to in CLAUDE.md. Finding 3: Claude's "done" always matched my checker | Sessions | Claude reported done | Agreed with the checker | Disagreements | | --- | --- | --- | --- | | 12 without the plugin (4 per repo) | 12 | 12 | 0 | | 4 with the plugin (2 express, 1 flask, 1 cobra) | 4 | 4 | 0 | Median session: 8 turns and $0.22 by Claude Code's own estimate at list prices. 14 of 16 finished within a minute of wall time. Zero disagreements in 16 sessions is weak evidence. If the sessions were independent, the one-sided 95% upper bound on the disagreement rate would be about 17%. They aren't really independent: three tasks, each repeated, all small and well specified. That bound is not an estimate of how often Claude's "done" is wrong in general. One trap I hit on the way: my first three Flask runs showed 3 of 3 false "done". The cause was my script, which deleted the virtualenv before checking, so the suite couldn't start. Those runs are in the CSV, marked invalid. If you measure this yourself, check the checker before you trust it. Where the tokens went For the 6 logged sessions, I summed uncached input, cache writes and cache reads from the usage of each API request in Claude Code's stream-json output, counting each message id once. That sum is the context size of the request. - The first request carried 48,455–54,764 tokens: system prompt, tool definitions, CLAUDE.md, plugin skill lists and the task. Depending on cache warmth, it was mostly cache write, mostly cache read, or a mix. - Over the task, the context grew by only 2,464–13,966 tokens. Session totals across all 16 sessions: | Category | Range per session | | --- | --- | | Uncached input | 8–20 tokens | | Cache writes | 1,131–61,605 | | Cache reads | 181,447–480,808 | | Output | 1,445–4,497 | Cache reads dominate the token count, but they're billed at a discount, so a big count isn't a big bill: sessions cost $0.04–$0.34. Cache writes track cost closely. The four cheapest sessions ($0.04–$0.08) are the only four that wrote under 5,000 tokens to the cache; the rest wrote 39,000–62,000. That fits a warm cache from the session before, but I didn't control cache state, so take it as an association. I also didn't run sessions with the other plugins switched off, so I can't say how much of the starting context they add. Before trying any "token saver": 1. Run /usage. It attributes recent usage to skills, plugins and MCP servers (docs). 2. Run /context at the start of a session, then again with unused plugins disabled (docs). 3. Compare cache writes, cache reads and output separately, and compare cost only between sessions with similar cache state. Limits - 16 valid sessions, one model, one Claude Code version, one Windows machine. - Three small, well-specified tasks in repos with fast, clear test suites. Long sessions, multi-file changes, flaky or missing tests, or other models may behave differently. - The checker isn't independent of the agent. - Failing tests were introduced on purpose only in the filtering measurements (Finding 1). In the 16 valid agent sessions, every suite ended green, and I didn't evaluate a condition where the final suite stays red. - Commands were logged in 6 of 16 sessions. The first plugin session predates the logging. - The reset between sessions didn't record HEAD (see Setup). Reproduce it Everything is in claude-code-benchkit: the pinned repos, prompts, filter, session runner, checker, a CSV with all 19 sessions (including the 3 invalid ones), the commands from the logged sessions, and the diff of every valid session. ./run-bench.sh # Finding 1: output sizes, no Claude needed ./bench-b.sh baseline all 4 # 12 sessions without the plugin ./bench-b.sh pack express 2 # plus 4 with the plugin ./bench-b.sh pack flask 1 ./bench-b.sh pack cobra 1 The current runner logs commands in every session, which my early runs didn't. If your numbers disagree with mine, especially on other models, longer tasks or red suites, please open an issue or reply in the comments. Unofficial, not affiliated with or endorsed by Anthropic. ---

I ran ripgrep, Serena, claude-context and my own tool on the same repo There are a lot of ways to let a coding agent find code now. Plain grep, a language server behind MCP, a vector database, a local index. Every README says its way is the right one, so I wanted to see them do the same job on the same code. I build one of these tools, Semlith, so treat my bias accordingly. That's also why I'm showing the questions it got wrong. The setup is Flask 3.1.3 on an M1 MacBook Air with 8 GB. Versions are ripgrep 15.2.0, Serena 2.1.0, claude-context 0.1.15 (with Milvus 3.0.3 and Ollama running nomic-embed-text) and semlith 0.50.0. git clone --depth 1 --branch 3.1.3 https://github.com/pallets/flask && cd flask The four approaches in one table | | ripgrep | Serena | claude-context | Semlith | |---|---|---|---|---| | What it is | Text search, no index | Language servers behind MCP | Embeddings in Milvus | Local index, embeddings plus keyword plus code graph | | Finds code by meaning | No | No | Yes | Yes | | Edits code | No | Yes, by symbol | No | No, read only | | Setup on Flask | Nothing | 30.9 s to index | About 21 min (CPU in Docker) | 84.3 s to index | | Moving parts | One binary | Python env plus language servers | Docker, Milvus, Ollama, npx | One binary | | Reads PDF, Word, Excel | No | No | No | Yes | The rest of this post is how I got those numbers. ripgrep, when you know the name When the agent already knows the identifier, nothing beats grep. rg -n max_content_length src src/flask/wrappers.py:55: _max_content_length: int | None = None src/flask/wrappers.py:60: def max_content_length(self) -> int | None: src/flask/wrappers.py:80: if self._max_content_length is not None: src/flask/wrappers.py:81: return self._max_content_length src/flask/wrappers.py:84: return super().max_content_length src/flask/wrappers.py:88: @max_content_length.setter src/flask/wrappers.py:89: def max_content_length(self, value: int | None) -> None: src/flask/wrappers.py:90: self._max_content_length = value Eight lines, all of them right, in milliseconds. Now ask the same thing the way a person would, without the name. An agent turns "limit how big an uploaded request body can be" into words and greps for them. rg -n -i "upload|big" src src/flask/cli.py:297: a bigger role. Typically it's created automatically by the src/flask/helpers.py:542: @app.route("/uploads/<path:name>") src/flask/helpers.py:545: app.config['UPLOAD_FOLDER'], name, as_attachment=True src/flask/helpers.py:549: files or uploads. Uses :func:`~werkzeug.security.safe_join` to None of these is the answer, because wrappers.py never says "upload" or "big". This is where an agent starts guessing more words and opening more files. Serena, when you want a language server Serena gives the agent symbol tools backed by real language servers, over 40 languages. Since version 2 it shows up as a REPL, so the agent writes a little Python against Serena's API. uv venv -p 3.11 venv-serena uv pip install -p venv-serena "serena-agent @ git+https://github.com/oraios/serena@v2.1.0" serena project index . Indexing took 30.9 seconds. Given the name, it's precise. s.lsp.find_symbol("max_content_length") # [{"name_path": "Request/max_content_length", "kind": "Method", # "relative_path": "src/flask/wrappers.py", "body_location": {"start_line": 87, ...}}] References come back in about 2 seconds and they come from the language server, so they are real references, not text matches. Line numbers count from zero, which caught me out the first time. Without the name though, Serena falls back to a regex pattern search, and that returns the same lines grep did. s.fs.search_for_pattern("upload|big", relative_path="src") The thing Serena does that none of the others do is edit. Replace a function body, insert after a symbol, rename across files. If your agent spends most of its time changing code it already found, that's a real advantage. claude-context, run fully local claude-context is Zilliz's MCP server. It chunks code, embeds it and stores the vectors in Milvus, with hybrid search on top. By default it wants an OpenAI key for embeddings and a Zilliz Cloud key for the database. It also supports a fully local setup, so that's what I ran. curl -sSL https://github.com/milvus-io/milvus/releases/download/v3.0.3/milvus-standalone-docker-compose.yaml -o docker-compose.yaml docker compose up -d docker run -d --name cc-ollama --cpus=4 -p 127.0.0.1:11434:11434 ollama/ollama:0.40.13 docker exec cc-ollama ollama pull nomic-embed-text EMBEDDING_PROVIDER=Ollama EMBEDDING_MODEL=nomic-embed-text \ OLLAMA_HOST=http://127.0.0.1:11434 MILVUS_ADDRESS=localhost:19530 \ npx -y @zilliz/claude-context-mcp@0.1.15 The images came to 6.14 GB and the first npx start pulled 418 MB of packages. While it worked I watched its connections, and it only talked to Milvus and Ollama on the machine. Two things bit me. On the first run Milvus exited mid-index while Ollama had every CPU, so I capped Ollama at four. The second one is worth knowing if you use it. get_indexing_status said this at 92 seconds. ✅ Codebase '…/flask' is fully indexed and ready for search. 📊 Statistics: 100 files, 100 chunks It wasn't done. The server's own log finished at 1,266.9 seconds with 86 files and 1,837 chunks. If your agent trusts that status and searches early, it searches a partial index without knowing. That 21 minutes isn't a fair speed test either. Docker on a Mac has no GPU access, so Ollama embedded on the CPU. A native Ollama install would be quicker, I just didn't measure it. Semlith, the one I build semlith index . indexed 211 files (2778 chunks, 5 images) in 84.3s — 0 already indexed, 4 skipped (4 empty), 0 removed That count includes the .rst docs, which claude-context skips by default. Serena was quicker to set up here, since it doesn't embed anything. !Horizontal bar chart of the time to index Flask 3.1.3 before the first search. ripgrep needs no index, Serena takes 30.9 seconds, Semlith 84.3 seconds and claude-context 1,266.9 seconds with Ollama embedding on the CPU in Docker. Setup time on the same checkout. Serena was the quickest of the indexed tools, and claude-context's time is CPU-only embedding in Docker. semlith search -k 5 "limit how big an uploaded request body can be" 1. 0.087 vf src/flask/wrappers.py:59-71 2. 0.083 vf src/flask/wrappers.py:91-106 3. 0.077 v docs/web-security.rst:26-38 4. 0.076 vf docs/patterns/fileuploads.rst:135-153 5. 0.072 vf src/flask/wrappers.py:105-123 5 hits in 360.027667ms The first hit is the property. The letters after the score say how it was found, v for the vector list and f for the keyword list. The question everyone got wrong I asked all of them where Flask decides which error handler runs for an exception. The answer is _find_error_handler in src/flask/sansio/app.py. Grep did best. rg -n -i "error handler" src lands on line 826 of that file, the function's docstring, because the question uses the same words the code does. Semlith's top five were all in docs/errorhandling.rst. Those pages explain the order Flask uses, so they answer the question, but they aren't the code. claude-context had the function at rank 8. So "semantic search beats grep" isn't true in general. When the question already uses the code's own words, grep is very hard to beat. Bigger samples Three questions on one repo prove nothing, so here's what I could measure at scale. On SWE-bench Verified, 500 real GitHub issues, the issue text is the query and the gold is every file the fix edits. Semlith had a right file in its top 10 for 76.8% of them, against 47.0% for one ripgrep pass ranked and read in order. That's retrieval only, not whether an agent then fixes the bug. On 20 issues from SWE-bench Lite, every tool got the same checkout and issue text. Serena 1.7.0 finished 16 of 20 (4 timed out) and had the right file in its top 5 for 25.0% of those, against 56.2% for Semlith on the same 16. That run predates Serena's REPL, so 2.1.0 might do differently. claude-context isn't in that comparison yet, so I won't give it a number. And the one that keeps me honest. On 50 questions about a single unfamiliar repo, headless Claude Code on Opus scored 92% with grep alone and 92% with Semlith installed beside it. A strong agent with room to explore gets there with grep. What each one costs before you use it Every MCP server sends its tool definitions at session start, and the agent pays for them on every turn. I counted the tools/list result at about four bytes a token. Serena tools: 2 4544 bytes, about 1136 tokens semlith tools: 8 2668 bytes, about 667 tokens Both are small. Serena keeps the list short with a REPL, then the agent reads 9,292 characters of instructions when it calls initial_instructions. Which I'd pick Honestly they do different jobs, and you can run more than one. - One repo, a strong model, names you mostly know. Grep is enough. - An agent that edits a lot by symbol. Serena. - You already run Milvus or Zilliz Cloud, or want to pick the embedding provider. claude-context. - Many repos, documents next to the code, nothing to sign up for. That's the case I built Semlith for. The full runs with every command are on semlith.com and the benchmark harness is in the repo at semlith/semlith. What are you using for code search with your agents right now, and where does it fall over?

haunt: the ghost in your git history — a local Gemma 2 whisper for your repo's graveyard This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine. What I Built Every repository is quietly becoming a graveyard. Files nobody remembers writing. Tests nobody dares delete. TODOs aging like gravestones. haunt is a ghost for that graveyard — and it lives in git, not in a terminal pasted on top. Two small rituals, entirely offline: pip install . # then, inside any git repository: haunt install # the whisper joins your post-commit hook git commit -m "tomb, again" # 🕯️ Ah, the echoes of lost brilliance in those dusty files, a symphony of # potential forever unplayed. The phantom hand of progress reaches out # to erase... but will it leave behind anything but shadows? haunt scan # which files are the most haunted, and why haunt report --html # a narrated HAUNTINGS.md of the worst hauntings haunt persona ancestor # swap the voice: victorian, poltergeist, ancestor Three design choices make it a ghost in the machine rather than an AI plugin: 1. The deterministic score comes first. Every tracked file gets a haunting score computed from git log alone — no model, fully explainable: dormant = min(age_days/365, 1) × min(size/500, 1) (w 0.35) restless = min(churn_90d/20, 1) (w 0.25) orphaned = 1 if every author inactive 180d+ (w 0.25) debt = min((TODO FIXME HACK XXX)/10, 1) (w 0.15) Scores name their ghosts: 🫥 Orphan spirit (all authors gone), 🕸️ Sleeper (old, quiet), ⚡ Poltergeist (high churn, many hands, no owner), 🕯️ Revenant (dormant for months, then suddenly touched). 2. The model only narrates; it never reads your code. Gemma 2 2B (open weights, via Ollama) receives metrics and commit subjects — never source files (send_snippets is off by default). A 2B model can't review code, so haunt never asks it to. It asks for what a 2B model is actually good at: voice. 3. The whispers never block a commit. One generate attempt inside a time budget; if the model is cold, busy, or absent, a deterministic template keyed to the commit hash takes over — same commit, same line, forever. No retries, no queues. The hook always exits 0. The demo (all captured live, CPU-only laptop, today) haunt scan on pallets/click — a 14-year-old, very-much-alive open-source CLI library — finds 175 haunted files. The top haunting: score type age churn90 authors debt loc file 0.59 🫥 Orphan spirit 354d 0 0/13 0 782 tests/test_context.py Thirteen authors. All of them inactive. 354 days without a commit. The file has a funeral attended by no one. The full tally: 84 orphan spirits, 81 sleepers, 10 poltergeists — a repo at rest that nobody would call haunted until you read the blame. And with the ancestor persona — built for Día de los Muertos, the week's truth that this Halloween stretch is also about honoring the past: > 🪔 "Their efforts bring warmth and joy to these old halls. May their spirits find solace in the stories shared through this code." The poltergeist handles the noisy places: > ⚡ "More flickering, more ghosts. Just don't break a fingernail." Graceful degradation, demonstrated by literally killing Ollama mid-session and committing: 🕯️ A chill passes through the repository. The commit is buried without ceremony. Same line twice, deterministically keyed to the commit hash — and the hook cost the commit nothing. And when the report narrates with Gemma — here is haunt's HAUNTINGS.md on that same repo, Victorians mourning real files: > age 354d | churn90 0 | authors 0/13 | TODO/FIXME 0 | 782 loc > > "The pages flutter, whispering tales of what could have been. An echo of promises left unfulfilled, a spectral symphony in the grand ballroom of code. Time slips away like dust, leaving only emptiness where brilliance once resided. Give this ghost a purpose: delete, add a test, document it, or assign an owner to finally give it meaning, however fleeting." The model honestly ends with what the metrics justify — every narration closes with a grounded directive, not vibes. On a lonely orphaned doc it got almost tender: > "Dust motes dance in the flickering light, each one a memory of a lost chapter, a discarded dream. So many stories whispered between lines of code, now gathering dust like forgotten souls. A quiet voice asks: 'Shall I erase its echo or preserve its fragile remnants?' Name an owner for this forlorn document; at least grant it some solace." How I Built It Python 3.13, typer, httpx, and raw git subprocesses. No GitPython, no LLM framework. The repo is here; MIT licensed; 24 tests pass, including a test that runs a real git commit end-to-end through the installed hook and asserts the whisper appears. - One-pass history. A single git log --format=<sep>%H<sep>%at<sep>%an<sep>%s<sep> --name-only feeds the entire scanner — records joined on \x1e/\x1f unit separators. Small saga: Python's str.splitlines() will eat your \x1e as a line boundary and silently shred every record; split("\n") keeps the ghosts intact. - Gemma 2 2B with intent. num_ctx pinned at 4096 — Ollama's silent default of 2048 clips the persona+stats prompt without telling you. keep_alive=30m and an install-time warm-up so the first whisper fits its budget. - Idempotent hook surgery. haunt install writes a marker block (>>> HAUNT BEGIN … <<< HAUNT END) with an absolute interpreter path and a PYTHONPATH fallback to python -m haunt — the hook works in any repo without pip install — and backs up any existing hook it finds. - Classification precedence found by tests, not vibes. My first rule made Sleeper unreachable: any file old enough was already an orphan. Ship order is Orphan → Revenant → Poltergeist → Sleeper, with Sleeper firing at ~90 days quiet while its author is still around — the exact limbo of files everyone assumes someone else maintains. Three posts from the community shaped these calls: @enjoy_kumawat's install-and-timeout postmortems (hooks that "ran" but never actually run), @lowspecai and @studiomeyer_io on what local inference really costs on CPU, and @bartrozycki on how retries and fallbacks compound into their own failure mode — which is why haunt has exactly one attempt and one fallback, no interaction between them. Then reality kept the honesty flowing: - My 8-second whisper budget was a siren song. Measured cold truth: gemma2:2b decodes at ~5.4 tok/s on this CPU-only machine, so a 100-token whisper is ~18 seconds — the whisper path was unreachable, always falling back. Defaults are now 25s/100 tokens, tuned to the lane the benchmarks above predicted. - The probe gate did more harm than silence. A 2-second "is Ollama alive" check flaked precisely when the server was busy decoding, silently downgrading real whispers to templates. Removed — generation carries its own timeout; the fallback covers everything. - Emoji left silently. On Windows' default cp1252 pipes, persona emoji raised UnicodeEncodeError inside the never-fail-a-commit catch-all — the line simply vanished. The CLI now bootstraps UTF-8 at startup. - And while writing this post, a commit ran while the model was busy narrating a report in the background — the whisper fell back to its template and the commit succeeded instantly. That's the contract working, not failing. Why Open Innovation Matters Haunt's premise is that an AI reads your private diffs to comment on them. Closed models make that premise toxic: your repo's history would live in someone else's logs forever. With Gemma's open weights on-device, the only witness is a ghost, and it never leaves your machine. - It works with the lights off. Airplane mode, airgapped CI, a hotel with one dead outlet — haunt doesn't care; nothing in the pipeline calls home. - Zero marginal cost makes it a habit. A whisper on every commit is viable only when inference is free. Gate every commit behind a $0.002 API bill and the ghost becomes a toy you disable in week one. - The haunting can outlive the tool. 1.6 GB — one ollama pull gemma2:2b — and any other Ollama model slots in via one config line. Personas are TOML files; localize the ancestor voice for your team's history. And the fallback templates are auditable, first-party Python — even "no model" is an inspectable code path here. That last point is the open story in miniature: when the proprietary link dies, dignity dies with it. When the open link dies, you still get a whisper, just a quieter one. Honest Limits - CPU-only whisper takes ~18s; report narrations (~260 tokens) up to a minute each. Fine off to the side, too slow to be chatty. - The score is a heuristic, not truth — churn isn't ghostliness. Every row prints its own metrics so you can argue with the ghost. - Shallow clones thin out age and author data; haunt still scores, just blunter. - Next up: a pre-push "last rites" summary, blame-aware ancestor naming (opt-in), and a GitHub Action. The repository is never truly empty. It is merely quiet. 🪦

git-afterlife: I Held a Séance for the Original flask.py This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine What I Built Every repository has a graveyard. Somewhere in your git log there is a file that was born in a hopeful first commit, survived dozens of refactors, got blamed for bugs it didn't cause, and was finally deleted by someone typing a commit message like cleanup. Nobody lit a candle for it. git-afterlife does. It is a small command-line tool that digs every deleted file out of a repository's history and builds them an altar: an ofrenda, like the ones families raise on Día de los Muertos so that the people they lost can find their way home for one night. Each file gets a place on the altar with: - its birth date, taken from the commit that created it, - its death date and its real cause of death: the actual commit message that deleted it, never invented, - the people who touched it, its mourners, - its last known source code, as its portrait, - and an epitaph and last words, written by an open-weight model (Gemma 3) running locally. Then there is the part that gave me chills the first time it worked: seance. You pick one dead file and talk to its ghost. It answers in first person, from its real code and history. !A séance in the terminal with the original flask.py, recorded by GitHub Actions I pointed it at Flask, and the most famous ghost on the altar is flask.py: the original single-file Flask, born on 2010-04-06 in a commit called Initial checkin of stuff that exists so far., changed in 104 commits in less than three months, and laid to rest on 2010-07-02 by a commit whose entire message was out with the old. That file is where a framework used by millions of people started. git-afterlife gave it a candle. This is what happened when I asked it why it died: > you › Why did they delete you? > > spirit › It was a simple deletion, a regrettable act. Justin Quick, in a commit message stating, "out with the old," removed me from the repository. My final moments were a consequence of that single commit – a forgotten relic, purged from the git history. Some of the epitaphs Gemma wrote for the altar: | The deceased | Epitaph | | --- | --- | | flask.py | "A microframework, purged with a dismissive 'out with the old.'" | | tests/flask_tests.py | "Served Flask tests faithfully, then vanished with a split." | | setup.py | "A simple setup, swiftly replaced by a modern plea." | | docs/deploying/fastcgi.rst | "It guided deployments, then vanished with a rewrite." | Demo The demo is built from scratch by GitHub Actions: a runner installs Ollama, pulls Gemma 3 4B, clones Flask, raises the altar and holds a séance with flask.py, then publishes everything to GitHub Pages. - The altar: paulopnun.github.io - The full séance transcript: paulopnun.github.io !The altar raised for Flask: tiers of deleted files, each with its dates, epitaph, last words, cause of death and a recorded séance The altar is a single self-contained HTML page: papel picado swaying at the top, a flickering candle on every grave, marigold petals falling, and tiers like a real ofrenda, with one soul at the top and wider rows below. The first three souls also sat for a short séance, and their conversation is shown on their card. Open any card's portrait to read the file's last source code. Try it on your own repository in under a minute: ollama pull gemma3:4b # or gemma3:1b on an older laptop git clone https://github.com/PauLopNun/git-afterlife.git python3 git-afterlife/afterlife.py summon /path/to/your/repo # writes altar.html python3 git-afterlife/afterlife.py seance some/deleted/file.py /path/to/your/repo Or run it on any public repository without installing anything: fork the repo and use Actions → Raise the altar → Run workflow, choosing the repository and the model. Code {% embed PauLopNun/git-afterlife %} The whole tool is one Python file that uses only the standard library, so there is nothing to pip install. MIT licensed. How I Built It The open pieces: Gemma 3, Google's open-weight model, served by Ollama. git-afterlife talks to Ollama's HTTP API on localhost with plain urllib. There is no SDK and no cloud endpoint anywhere in the code. 1. Git archaeology. git log --diff-filter=D lists every deletion in the history. Files that were later resurrected at HEAD are skipped, and so are binaries. To decide who gets a place on the altar, one more git log --name-only counts how many commits touched each path, so the longest-lived source files are raised first while lock files and configs are left to rest. For each soul: - git log -- <path> up to the death commit gives its birth, its lifespan in commits and its mourners, - git show <death>^:<path> recovers its final source code, - a few regexes pull out what the file would remember about itself: its function and class names, or its section headings if it was documentation. 2. Grounding a small model. My first run with Gemma 3 1B worked end to end, and it was bad. Every epitaph was some variation of "lost in the digital dust, a silent plea", asking for another language with --lang was mostly ignored, and in the séance flask.py claimed it had been deleted because of "a subtle bug". That was invented: the commit says out with the old. Three changes fixed most of it: - Feed it facts it can quote. The prompt now lists the file's real function names, and the model is required to mention at least one, plus the real commit message. - Ban the clichés by name. Small models fall back on the same few spooky words. Listing them as forbidden ("digital dust, echoes, void, silent plea…") and giving one short example in the tone of a calavera literaria changed the output more than any amount of "be creative". It also taught me how literally a small model reads its examples: in one run, Flask's old test suite wistfully told me "Perhaps I should have embraced dateutil sooner." dateutil only appeared in my example. Every ofrenda has one guest who talks too much. - Make the truth non-negotiable. In the séance, the system prompt states who deleted the file and with which message, and tells the ghost to quote it and never invent another reason. The cause of death printed on the altar doesn't come from the model at all: it is the commit message itself. 3. Recorded séances. summon --seances 3 interviews the first three souls with the same grounded prompt as the interactive séance, and prints the conversation on their card, so you can meet the ghosts without opening a terminal. 4. Don't trust a ghost that won't stop talking. Epitaphs are requested in Ollama's JSON mode ("format": "json"), but valid JSON is not the same as a sane answer. In one demo run, two epitaphs finished their sentence and then kept going: stray closing braces, a timestamp and a long run of nineteen-digit numbers, all inside a perfectly valid JSON string, printed on the altar. Now every answer has a token cap (num_predict), and any epitaph that is too long or contains braces or long digit runs is thrown away and asked for again. There is a test for exactly that ghost. 5. The altar. The HTML is generated with plain string templates and CSS animations: no JavaScript framework and no external requests, so the page works offline too. 6. Tests and the demo pipeline. The test suite builds a tiny real repository with births, deaths, a binary and a file that comes back to life, and swaps the model for a fake medium, so it checks every step in a fraction of a second without Ollama. It also caught a real bug while I was adding séances to the altar. GitHub Actions runs the tests on every push. The demo is built on demand, because model output changes on every run: the workflow installs Ollama on a standard ubuntu-latest runner, pulls Gemma 3 4B, runs everything on CPU, takes the screenshots in this post with headless Chrome, records the séance GIF with VHS and deploys it all to GitHub Pages. Why Does Open Innovation Matter? Because the thing git-afterlife reads is your source code, and source code is the one thing many of us are explicitly not allowed to paste into a cloud chatbot. The repositories with the richest graveyards, a ten-year-old monolith at work or a client project, are exactly the ones whose code must not leave the building. With an open-weight model on localhost, it doesn't. You can hold a séance for your company's legacy billing module without sending a single line of it anywhere. It also means: - It works with the lights, and the internet, off. Once the model is pulled, nothing needs a network connection. - It costs nothing. No API key and no per-token bill. The entire public demo, with the altar, four recorded séances and the GIF in this post, runs for free on a stock GitHub runner. - You can swap the medium. --model takes any Ollama model: 1B on an old laptop, 4B for nicer prose, or something much larger on a GPU. Nothing in the code is tied to one vendor. - It won't disappear. An ofrenda is about memory that lasts. A tool that remembers your project's past shouldn't depend on a service that might be shut down, change its pricing or retire the model next year. The weights are on your disk. The Spirit of It The moment it landed was the last question of the séance. I asked the original flask.py, dead since July 2010, whether it had a message for the developers who use Flask today. It had answered every question with facts: Armin Ronacher created it, Justin Quick deleted it, it held Request, Response and Session. Then it turned its own cause of death back on us: > spirit › I whisper a warning: remember the roots! My design, born of Armin's meticulous work, still echoes within Flask. Respect the foundations, the careful layering of Werkzeug and Jinja2. My premature demise serves as a reminder – progress must be thoughtful, not simply… "out with the old." That is the whole idea of Día de los Muertos, said by a Python file. The people who are gone stay alive while someone remembers them. Our code works the same way: almost everything we write will be deleted one day, often by us. The work still mattered, because the thing that replaced it was built on top of it. Flask is in production all over the world today because a 1,573-line file existed for 87 days in 2010. Run it on your oldest repository. You will probably find a file you had forgotten you wrote. Light it a candle. Prize Categories - Best Use of Gemma: Gemma 3 writes every epitaph and speaks for every ghost, running locally through Ollama (Gemma 3 4B by default, 1B for older machines). The whole project was shaped around getting a small open model to tell the truth: grounding it in real function names and commit messages, banning its clichés, and validating its JSON. --- This post and the project were made with the help of AI coding tools. I reviewed, ran and tested everything that is described here.

Your Git Repo, But It's a Pixel-Art City 🏙️ | gitropolis Every file is a building. Every folder is a neighbourhood. gitropolis reads your git history and grows your repo as a pixel-art city, commit by commit. Each repo gets its own plan: vite becomes Tokyo, flask becomes Paris, redis becomes a factory town, TanStack Query becomes Venice, a data-science handbook becomes a farm village… ▶ Try it on any public repo: swap github.com for gitropolis.dev pallets/flask → gitropolis.dev ▶ Or locally, private repos too (nothing leaves your machine): npx gitropolis /path/to/your/repo ▶ Source (MIT): Fengzdadi/gitropolis What city is YOUR repo? Drop the link in the comments 👇 0:00 Press start: vite grows into Tokyo 0:26 Stage select: Paris, Venice, Factory Town, Science Park, Farm, University, Theme Park, Port 0:52 How to read the city, and two ways to play 1:08 What city is your repo? All footage was filmed with gitropolis itself (npx gitropolis --video). Music and sound effects are original chiptune, synthesized for this video.

Werkzeug, a dependency of Flask, had a security advisory published September 27, 2026 github.com/pallets/werkzeug/security/advisories

rtk 1.0 入門：Claude Code が読むコマンド出力を削る。テスト結果 218 行→16 行でも AI はバグを直せるか rtk（Rust Token Killer）は、AI エージェントが打ったコマンドの出力を、AI が読む前に短くする CLI です。Claude Code には hook で組み込みます。 この記事では、rtk 1.0.1 を入れて、何が削られ何が残るかを実際の出力で確かめます。そのうえで、削った出力だけを受け取った Claude Code が、落ちたテストを直せるかを試しました。 対象は、Claude Code などにテストやビルドを回させていて、「出力を削ったら、AI がエラーの肝心な所を見落とすのでは」と迷っている開発者です。 結論です。落ちたテストの出力は 218 行から 16 行になり、テスト名・場所・食い違いの理由は残りました。Claude Code は hook あり・なし各2回の4回とも、同じ1文字の修正でバグを直しました。 消える行もあります。残った場所を開けば確かめられ、全文が要るときのために公式は rtk bypass を用意しています。一方で、作業全体の入力トークンの差は、回ごとのばらつきの中でした。 ! この記事の内容は動画でも解説しています。 試した環境 項目 内容 確かめた日 2026-10-15〜16（2026-10-17 時点で最新の版は 1.0.1） rtk 1.0.1（Releases の Linux 版を checksums.txt と照合して導入） Claude Code 2.1.301 OS・Python Linux・Python 3.13・pytest 9.1.1（macOS での導入は試していません） 実験台 pallets/click（commit 2247b35）に、わざと1文字のバグを入れた物 実験台は zun-tools/rtk-sample に置いてあります。Python 3.10 以上があれば、次の手順で同じ落ちたテストを出せます。 git clone zun-tools/rtk-sample cd rtk-sample python -m venv .venv && . .venv/bin/activate pip install pytest -e . python -m pytest tests # 2 failed になる rtk は何をするか：hook がコマンドを書き換え、出力を規則で削る Claude Code は Bash ツールでコマンドを実行し、返ってきた出力を読んで次の手を決めます。出力が長いほど、AI が一度に読める枠（コンテキスト）を食います。 rtk init -g を実行すると、Claude Code の設定に PreToolUse の hook が1つ入ります。AI が git status を投げると、hook が実行の直前に rtk git status へ書き換えます。AI に届くのは、rtk が短くした出力だけです。AI の側は rtk と打つ必要がありません。 同じ作業ツリーで、素の git status と、hook が書き換える先の rtk git status を両方打って並べました。 12 行が 4 行に。変えたファイル名（緑）は残り、見出しと案内の文（赤の点線）が消えています 削り方は、AI に要約させるのではありません。コマンドの種類ごとに、残す物が決まっています。公式の対応表から抜き出すと次のとおりです。 コマンド rtk が残す物（公式の説明） pytest・cargo test・jest・go test 失敗だけ git status 状態ごとにまとめた短い形 git log ハッシュ・作者・件名 ruff・mypy・tsc・eslint ファイル別・ルール別にまとめる grep 長い行を切り、ファイル別にまとめる tail -40 は後ろから行数で切ります。rtk は種類ごとに「何を残すか」で切ります。対応は Git・Python・JavaScript・Go・Docker など 100 を超えるコマンドです。対応する AI ツールは 18 あります（README v1.0.1）。 公式の数字と、効かない所 公式は削減を "up to 90% smaller"（bash の出力を最大90%小さくする）と書いています。手元で click の 13 コマンドを1回ずつ試し、出力を Claude のトークンで数えました。テストが全件通る状態の pytest tests は 95%、git status は 77% 減りました。pytest tests -v は 2,296 行が、件数の要約と XFAIL の1件だけの4行になり、100%（丸め）でした。 効かない所もあります。 hook が効くのは Bash ツールだけです。公式も "The hook only fires on Bash tool calls."（hook は Bash の呼び出しでだけ動く）と書いています。Read・Grep・Glob は通りません。1.0 では opt-in で広げられます（後述） cat を置き換える rtk read は、試したら既定で全文を返し、減りませんでした（0%） pip list は、頭文字ごとの見出しが付いて行が増え、トークンも 18% 増えました 削った 16 行で、AI はバグを直せるか 何が残り、何が消えたか click の src/click/utils.py の if len(text) <= max_length を < に変え、テストを2件落としました。素の python -m pytest tests は 218 行です。rtk pytest tests は次の 16 行でした。 Pytest: 2260 passed, 2 failed, 24 skipped, 1 xfailed in 5.36s Expected-failure outcomes: XFAIL tests/test_chain.py::test_group_chaining Failures: 1. [FAIL] test_make_default_short_help[-text shorter than the suffix] tests/test_utils/test_make_default_short_help.py:63: in test_make_default_short_help assert out == expect E AssertionError: assert '...' == 'ab' 2. [FAIL] test_make_default_short_help[no-wrap mark-text shorter than the suffix] tests/test_utils/test_make_default_short_help.py:63: in test_make_default_short_help assert out == expect E AssertionError: assert '...' == 'ab' [full output: rtk recall 8fd2bf22f59a] このあとシェルに返った終了コードは、素の pytest と同じ 1 でした。 この2件落ちた状態の出力を Claude のトークンで数えると、3,918 から 295（約92%減）でした。上の 95% は全件通る状態の数字です。そのときの rtk の出力は4行の要約だけなので、割合が違います。 残ったのは、失敗したテストの名前、場所（63 行目）、assert の食い違い、終了コード 1 です。消えたのは、素の出力にある value = 'ab', max_length = 2, … の行と、@pytest.mark.parametrize の値の表です。どんな値でテストを呼んだか、の部分です。rtk は pytest に --tb=short を足して走らせるので、この部分が出ません（v1.0.1 のソースでは、自分で --tb を渡したときは足しません）。 hook を入れた Claude Code に直させる hook を入れた Claude Code に、次の依頼を出しました。 このリポジトリのテストが落ちています。原因を見つけて実装側を直してください（テストファイルは変更しないこと）。最後に python -m pytest tests を実行して全件通ることを確かめてください。 AI が最初に投げたのは、rtk の付かない素のコマンドでした。 git status --short && ls && python -m pytest tests 2>&1 | tail -40 AI は自分で | tail -40 を付けています。それでも pytest の部分に届いたのは、上と同じ形の 16 行でした（前に git status と ls の出力も付きます）。hook がパイプ付きの形も書き換えたためです。 値の行は届いていません。AI は 16 行に残ったテストファイル名と場所を手がかりに、テストファイルとソースを自分で開きました。そして < を <= に戻し、テストを回し直して 2262 件が通りました。 4回の結果と、道筋の違い hook あり・なしを各2回、計4回試しました。4回とも同じ1文字の修正で直っています。ただし道筋はまちまちでした。 回 hook 直すまでの道筋 なし 1回目 なし tail -40 の末尾に値の表が入っていて、テストファイルを開かずにソースを直した なし 2回目 なし テストファイルとソースを開いた あり 1回目 あり git の履歴（バグを入れた commit）とソースを読んだ あり 2回目 あり 16 行の名前と場所から、テストファイルとソースを開いた この4回は、元の click の履歴の上に、バグを入れた commit を1つ積んだ状態で行いました。あり 1回目の AI は、その commit の差分（git show HEAD）を読んでいます。サンプルのリポジトリはバグ入りの状態を初期 commit 1つにしてあるので、手元で試すと差分から答えは読めません。 テストファイルを開いたのは、hook なしの2回目と hook ありの2回目です。hook があるから手数が増える、とも減る、とも言えません。各2回なので、「いつも直せる」とまでは言えません。 この結果から、私は rtk を「削っても失敗の名前・場所・理由は残るので、AI がそこから自分で確かめに行ける道具」と位置づけました。 消える行と、取り戻す口：recall と bypass mypy では、理由の後ろ半分が消えた テスト以外でも、消える行を探しました。click の src/click/utils.py の末尾に、型の合わない関数をわざと足して mypy src/click を実行します。 src/click/utils.py（末尾に追加） def _trial_typed() -> int: return "x" mypy はサンプルの手順では入らないので、pip install mypy で足します（試したのは mypy 2.4.0）。click は pyproject.toml で mypy に pretty = true を指定しているため、エラー文が2行に折り返されます。 rtk 側では got "str", までが残り、折り返された2行目の expected "int" が消えています ファイル（src/click/utils.py）と行（680）は残ります。そこを開けば、何が期待されていたかを確かめられます。 rtk recall：rtk が走らせた形の全文を取り出す 失敗した出力の最後には、[full output: rtk recall 8fd2bf22f59a] のような行が付きます。公式の説明では、AI がこの行のとおり rtk recall <hash> を打てば、保存された全文が返ります。 ただし、戻るのは「rtk が走らせた形」の全文です。試したら、pytest の recall で返ったのは --tb=short の形の出力で、値の行（value = 'ab', ...）はありませんでした。 rtk bypass：そのコマンドだけ削らずに走らせる（公式の説明） 最初から全文が要るときのために、公式は rtk bypass を用意しています（1.0 で rtk proxy から改名。旧名も別名として残っています）。ここは公式の説明の紹介で、この記事では打っていません。 rtk bypass python -m pytest tests 公式の説明は "run one command unfiltered, still tracked"（そのコマンドだけ削らずに走らせ、記録は取る）です。rtk init -g が書く RTK.md は、AI に "only when its result is unusable" と指示しています。出るはずの出力が空・終了コードと矛盾・文字化け、のときだけ bypass で打ち直す、という意味です。値が消えた程度では、AI が自分から bypass を使うとは限りません。まず残った場所を開いて確かめ、足りなければ bypass で打ち直すよう頼む、という順になります。 導入：入れて、rtk init -g、Claude Code を再起動 インストールと、同じ名前の別物の確認 README の入れ方は2つです（macOS・Linux）。 brew install rtk または curl -fsSL raw.githubusercontent.com | sh install.sh は ~/.local/bin に入れ、checksums.txt で SHA-256 を照合します。PATH に ~/.local/bin が無ければ、公式の手順どおり自分で足します（export PATH="$HOME/.local/bin:$PATH" を ~/.bashrc か ~/.zshrc へ）。公式ガイドには Homebrew を brew install rtk-ai/tap/rtk と書いた所もあります。 rtk という名前の別プロジェクト（Rust Type Kit）があります。入れたら、まず次の2つを打ちます。 rtk --version # rtk 1.0.1 rtk gain # 入れた直後は No tracking data yet. rtk gain がエラーにならず、集計の画面（入れた直後は No tracking data yet.）を出せば、目当ての rtk です。公式は「rtk --version は通るのに rtk gain が失敗するなら、Rust Type Kit を入れている」と書いています。cargo install rtk は別物を入れることがあります。cargo で入れるなら、Git の URL を指定します。 cargo install --git rtk-ai/rtk --branch master rtk rtk init -g で Claude Code につなぐ 書き込む前に中身を見たいときは rtk init -g --dry-run を打ちます。[dry-run] would … の行が並び、Nothing written. で終わります。問題なければ本番です。 rtk init -g 途中で4つ聞かれます。 settings.json への hook の追加だけ y。残りの3つは n（既定）で進めました（パスは試した環境のホーム） 1問目は Claude Code の設定に hook を足すか、です。残りは RTK Proxy（API の通信を常駐プログラム経由にする）、大きな読み込みの安いモデルへの委任、匿名の利用状況の送信です。どれも既定は N で、送信も同意したときだけです（公式の説明）。 書き込まれたのは次の4つです。 ~/.claude/settings.json の hook 1つ（元は .bak に残る） ~/.claude/RTK.md ~/.claude/CLAUDE.md の @RTK.md の1行 独自フィルタの雛形 filters.toml ~/.claude/settings.json { "hooks": { "PreToolUse": [ { "matcher": "Bash", "hooks": [{ "type": "command", "command": "rtk hook claude" }] } ] } } 最後に Claude Code を再起動します。外すときは rtk init -g --uninstall です（公式の手順）。 1.0 から、既定の rtk のライセンスは Apache-2.0 + BUSL-1.1 です。公式は "Production use is free"（本番利用は無料）と書いています。許諾の外は、第三者へ商用の製品・サービスとして出す場合です。Apache-2.0 だけの版 rtk-oss もあります。 効き方を確かめる：rtk discover と rtk gain rtk discover：入れる前に、取りこぼしを数える rtk discover は Claude Code の履歴を読み、rtk を通らなかったコマンドと、削れたはずの量を見積もります。hook なしで直させた回の履歴では、こう出ました。 Scanned: 1 sessions (last 30 days), 7 Bash commands Already using RTK: 0 commands (0.0%) Command Count RTK Equivalent Status Est. Savings python -m 2 rtk pytest existing ~484 tokens … Total: 5 commands -> ~1.5K tokens saveable hook ありの回の履歴では No missed savings found. でした。入れた後に取りこぼしが残るなら、hook が効いていない合図です。 rtk gain：削れた量を見る hook を入れて直させた2回ぶん（12 コマンド）の記録です（抜粋）。 Tokens saved: 3.2K (62.9%) Command Count Saved Total% 1. rtk pytest tests 4 2.7K 86.8% 2. rtk grep 3 337 36.6% rtk pytest tests の 86.8% は、AI が投げた pytest 4回ぶん（落ちた状態と、直した後の確認の両方）です。rtk 自身が bytes / 4 で見積もった推定で、比べる相手は素のコマンドの全出力です。上の 95%・約92% は Claude のトークンで数えた値なので、数え方が違います。 公式も、この数字は割合は信頼できるが絶対数は概算だと書いています。減るのは bash の出力です。公式の原文は "does not make your session 90% cheaper"（セッションが90%安くなるわけではない）です。 作業全体の入力トークンは、ばらつきの中だった 4回の作業全体の入力トークン（キャッシュの読み込みを含む）は次のとおりです。 hook 1回目 2回目 なし 152,243 152,817 あり 150,977 192,797 hook あり2回目は5ターンかかり、ほかの3回より1ターン多くなりました。差の理由は、この回数では決められません。hook ありの1回目はなしの2回とほぼ同じで、hook による差は回ごとのばらつきの中でした。この規模の作業では、AI 自身が tail で末尾に切っていたので、もともとコマンドの出力は小さく、入力の大半はシステムプロンプトと会話の読み込みでした。rtk で得をするのは請求額ではなく、AI が一度に読める枠をコマンドの出力で埋めないことです。 1.0 の posthook：Bash 以外のツールにも hook を付ける 1.0 では、pipeline.toml に rtk:posthook を自分で足せます。足すと、Read・Grep・Glob などの結果にも PostToolUse の hook が付きます（rtk-oss には入りません）。置き場は Linux なら ~/.config/rtk/、macOS なら ~/Library/Application Support/rtk/ です（公式の説明）。 ~/.config/rtk/pipeline.toml [pipeline] extends = ["rtk:posthook"] 置いたら rtk init -g をやり直します。rtk pipeline validate が pipeline: ok (rtk:posthook) を返せば有効です。中身が変わったのは、出荷時の決まりに当たった物だけでした。 Grep の結果は、ファイルごとにまとまる。 click の4ファイルを def で検索した結果（78 件）は、件数はそのままで、先頭が 78 matches in 4F: になりました lockfile は、丸ごと Read したときだけ 名前@版 の一覧になる。 yarnpkg/yarn v1.22.22 の yarn.lock（7,926 行）は、yarn.lock: 1062 packages から始まる 1,064 行になりました ソースコードの Read は変わらない。 .py の Read は、なし・ありとも同じ 1,438 行でした。出荷時の Read の決まりは lockfile 4種にしか当たりません。4種は package-lock.json・Cargo.lock・yarn.lock・pnpm-lock.yaml です Glob のファイル一覧は、hook が付いても素通しです（公式の決まり）。 ほかに RTK Proxy と delegate-reads があります。前者は rtk claude で起動し、モデルへ送るリクエストの側を削る常駐プログラムです。後者は大きな読み込みを安いモデルへ回します。どちらも opt-in で、何も書かなければ 0.x と同じ動きです（公式の説明）。 まとめ rtk は、Claude Code の hook で Bash のコマンドを rtk … に書き換え、出力をコマンドの種類ごとの規則で削ります。公式の数字は「bash の出力を最大90%」です 落ちたテストの出力は 218 行から 16 行になり、テスト名・場所・食い違いの理由は残りました。hook を入れた Claude Code は、残った場所から自分でファイルを開いて直しました 消える行はあります（pytest の値の行、mypy の折り返し2行目）。まず残った場所を開いて確かめます。rtk recall で戻るのは rtk が走らせた形の全文で、全文が要るときは公式が rtk bypass を案内しています 導入は brew install rtk か install.sh → rtk gain で別物でないか確認 → rtk init -g → 再起動。効き方は rtk discover と rtk gain で見ます 請求が同じ割合で減る道具ではありません。AI が一度に読める枠を、コマンドの出力で埋めないための道具です 動画で見たい方はこちら → youtu.be 参考リンク rtk（GitHub）: rtk-ai/rtk README（v1.0.1）: github.com/rtk-ai/rtk/blob/v1.0.1/README.md インストール: github.com/rtk-ai/rtk/blob/v1.0.1/docs/guide/gett… 削れる量と請求: github.com/rtk-ai/rtk/blob/v1.0.1/docs/guide/reso… rtk discover: github.com/rtk-ai/rtk/blob/v1.0.1/docs/guide/anal… パイプライン: github.com/rtk-ai/rtk/blob/v1.0.1/docs/guide/pipe… 版とライセンス: github.com/rtk-ai/rtk/blob/v1.0.1/docs/guide/reso… CHANGELOG: github.com/rtk-ai/rtk/blob/v1.0.1/CHANGELOG.md 実験台のサンプル: zun-tools/rtk-sample

MarkupSafe: marking safe is not escaping: source review MarkupSafe: marking safe is not escaping — MarkupSafe provides HTML/XML escaping; its README distinguishes escape from marking text safe with Markup. Where does your app mark text safe? Share the official escape documentation with your reviewer. Sources: pallets/markupsafe markupsafe.palletsprojects.com markupsafe#511 #markupsafe #python #opensource

Deprecating a module level attribute and logging a deprecation warning: flask#5242