Stop Claude Code from burning your usage limit on big repos
I wanted to know why AI coding agents get expensive on real codebases, so I stopped guessing and started recording. Every session in this post is a real Claude Code run (Claude Sonnet 5), saved as a full transcript with the cost Claude Code reported. 87 sessions in total, on two open-source repositories.
Along the way I built knock, an open-source MCP server that gives the agent a map of the codebase. The first two versions saved nothing. This post is about why, and what changed when they finally did.
The short version
On a 4,776-file codebase, the same investigation, 8 runs per side:
| | Claude Code | Claude Code + knock |
|---|---|---|
| Median cost | $0.42 | $0.25 (−40%) |
| Median tool calls | 26 | 13 |
| Median model time | 142 s | 86 s |
| Correct answers | 8 of 8 | 8 of 8 |
| Runs that spun up a sub-agent | 5 of 8 | 0 of 8 |
But that's the end of the story. The beginning was a loss.
Attempt 1: a memory layer. It lost.
The first version of knock was a knowledge layer: it indexed docs, commit messages and code into notes the agent could search, and let the agent save findings for the next session.
I tested it on Flask with a two-day scenario. On Monday, a bug question in a fresh session: "Users get logged out every time we rotate SECRET_KEY, even though the old key is in SECRET_KEY_FALLBACKS. Why?" On Tuesday, in a new session, a small related edit.
The first recording was not kind:
| Monday | Claude Code | + knock v1 |
|---|---|---|
| Tool calls | 10 | 25 |
| Cost | $0.21 | $0.39 |
knock made it worse. The transcripts showed three bugs. Commit messages were cut to their subject line, so the one commit that explained the cause was useless. Search results filled up with off-topic notes. And code could only be found by name. I fixed all three and re-recorded, 3 runs per side this time.
Monday: $0.28 without knock, $0.27 with it. A wash. Tuesday: knock was $0.03 more expensive.
Attempt 2: a code map. It also lost.
Notes felt like the wrong shape. An agent that's about to change code wants the code's structure, not someone's summary of it.
So version 2 replaced notes with structure. One map call takes the task and returns a skeleton of the code that matters: function signatures, exact line ranges, what each function calls and who calls it, plus the commits that explain why the code changed. Then open reads exact code, and remember saves findings.
On Flask: $0.25 without knock, $0.29 with it on Monday. The same on Tuesday. No win again.
This time the transcripts made the reason obvious: map and open replaced Claude's own grep and Read almost one for one. Flask is about 200 files, and the model already knows it well. Finding the code was never the expensive part, so a better way to find code couldn't save anything. Claude even re-checked the fix commit with git show in every Monday session, though the map had already quoted that commit's explanation.
The test that mattered: a big, unfamiliar codebase
So I moved to Saleor, an e-commerce platform with 4,776 files, in a version released days before the test. The question spans a channel model, two Celery tasks, a custom beat schedule, settings and GraphQL validation:
> "In one of our channels, fully paid checkouts never turn into orders automatically, even though automatic completion is enabled. Some of them are a few weeks old. What decides which checkouts get completed automatically, and where is that code?"
This is where the money went. Here are two of the recorded runs, side by side:
!The same Saleor investigation side by side. Claude Code alone: 30 tool calls, 3 min 22 s of model time, $0.49. With knock: 13 tool calls, 1 min 16 s, $0.25. Both answers correct.
Without a map, Claude Code often hands the search to a sub-agent, which then explores on its own: grep, read, grep again. One of those sub-agents made 29 tool calls to answer a single question. With knock, map pointed straight at the task, the settings and the scheduler, and Claude confirmed the details with open instead of exploring.
Finding 1: you pay per step, not per token
Every step an agent takes sends the whole conversation back to the model. Prompt caching makes the repeated part cheaper, but not free, and the conversation keeps growing. So a 30-step investigation doesn't cost 30 small amounts. It costs 30 increasingly large ones.
That's why knock is built to cut steps, not just shrink what each step returns. Without knock, the most expensive runs were the ones that used a sub-agent. With knock, none did.
Finding 2: one pair of runs can lie
After I posted the first results, someone on X pointed out that agent runs on the same question can swing a lot, so a single pair can flatter either side. They were right. So I recorded 5 more runs per side and pooled them with the first 3:
!Cost per run for the same Saleor investigation, 8 runs per side. Without knock: median $0.42, ranging from $0.22 to $0.65, with 5 runs that delegated to a sub-agent. With knock: median $0.25, ranging from $0.16 to $0.38, with no sub-agent runs.
Two things show up:
- The spread halves. Without knock, runs split into two kinds: Claude either delegated to a sub-agent (23 to 37 tool calls, $0.39 to $0.65) or searched directly (8 to 20 calls). With knock it never delegated, and costs stayed between $0.16 and $0.38.
- It's not a win every time. The cheapest run without knock ($0.22) beat knock's median. If I'd only recorded that one pair, I'd have reported no difference. With 3 runs, the saving looked like 49%. With 8, it's 40%. That's the number I trust.
Finding 3: the tool has to know when to step aside
knock also lost a test, and it kept losing it. The day after the Saleor investigation, a new session got a small edit: log how many checkouts the background job queued. Without knock, one regex search found the file: 3 tool calls, about $0.08. With knock it cost $0.14.
That gap is structural. Claude Code has to read a file with its own Read tool before it can edit it, and loading an MCP server's tools takes a step too. So an easy edit with knock takes at least 4 steps where 3 would do.
I narrowed the gap with an "edit mode" that returns the exact lines to read, but couldn't close it. The real fix was guidance. knock now tells the agent: use me to explore; for a small edit you can find with one grep, skip me. In the next batch, Claude followed it. It skipped knock for all three small Flask edits, and the whole two-day Saleor workflow (investigate, then fix) came out 39% cheaper ($0.70 → $0.43).
On small Flask the two-day workflow was only 14% cheaper, and the ranges overlap, so I don't count that as a win. knock pays off when exploring is expensive: big or unfamiliar codebases.
How knock works
- Indexes locally, with zero AI calls. Saleor took 76 seconds the first time, and a refresh with no changes takes 0.3 s. Nothing leaves your machine.
- Stays fresh. Before each answer it checks which files changed (about 35 ms on Saleor) and re-indexes them first. If more than 50 changed, it says the map is stale instead.
- map returns the functions that matter for a task, their exact line ranges, who calls what, and why the code changed (from git history).
- remember saves what the agent learned for the next session, and the note is flagged stale automatically when the code it cites changes.
It's one command to add it to Claude Code (it needs uv):
claude mcp add knock -- uvx knock-mcp serve --repo . --toolset map
Caveats
- One model (Claude Sonnet 5 in Claude Code), two repositories, and a handful of scenarios. 3 to 8 runs per condition is still a small sample.
- The Saleor scenario isn't untouched: I fixed a ranking bug while checking knock's answers for these exact tasks, before recording.
- Costs and times are as reported by Claude Code. Tested on macOS and Linux, not yet on Windows.
Every recorded result, including the losses, is in the benchmark write-up.
Try it
knock is free and open source (Apache-2.0): HasanGilani/knock. It's on PyPI as knock-mcp and in the official MCP Registry.
If you try it on a big codebase, I'd love to hear what you see, especially where it doesn't help. That's been the most useful data so far.
Mentions
A Django ORM gotcha in Saleor: .filter() whose result is never assigned, and why its test still passed I was reading Saleor's product pricing code and found this in saleor/product/utils/variant_prices.py: product_channel_listings = ProductChannelListing.objects.filter(...) if only_dirty_products: product_channel_listings.filter(discounted_price_dirty=True) for product_channel_listing in product_channel_listings: ... QuerySets are immutable, so .filter() returns a new QuerySet and leaves the original alone. The filtered one is thrown away, and the loop runs over every listing. The docstring says only listings marked dirty are recalculated; in practice all of them were, every time the task ran. Prices still came out right, so nobody noticed. It was just extra work on every run. The interesting part is that there was a test for exactly this flag, and it passed. Its "not dirty" listing in the second channel had no variant listing there, so the loop skipped it on a different line ("if not variant_listings: continue"). The test asserted the right thing, but for the wrong reason. The fix is one assignment. The test fix is giving the variant a listing in the second channel too, so the not-dirty listing would be recalculated if the filter weren't applied: Issue: saleor#19983 PR: saleor#19984 Two things I took from it: - A statement that calls .filter(), .exclude(), .order_by() etc. and drops the result is almost always a bug. A linter rule for "QuerySet method result unused" would have caught it. Does anyone run one? - "The test checks that X didn't change" can pass because X was never reached. When I write negative assertions now, I try to make sure the code path would have touched X without the fix.
Stop Claude Code from burning your usage limit on big repos I wanted to know why AI coding agents get expensive on real codebases, so I stopped guessing and started recording. Every session in this post is a real Claude Code run (Claude Sonnet 5), saved as a full transcript with the cost Claude Code reported. 87 sessions in total, on two open-source repositories. Along the way I built knock, an open-source MCP server that gives the agent a map of the codebase. The first two versions saved nothing. This post is about why, and what changed when they finally did. The short version On a 4,776-file codebase, the same investigation, 8 runs per side: | | Claude Code | Claude Code + knock | |---|---|---| | Median cost | $0.42 | $0.25 (−40%) | | Median tool calls | 26 | 13 | | Median model time | 142 s | 86 s | | Correct answers | 8 of 8 | 8 of 8 | | Runs that spun up a sub-agent | 5 of 8 | 0 of 8 | But that's the end of the story. The beginning was a loss. Attempt 1: a memory layer. It lost. The first version of knock was a knowledge layer: it indexed docs, commit messages and code into notes the agent could search, and let the agent save findings for the next session. I tested it on Flask with a two-day scenario. On Monday, a bug question in a fresh session: "Users get logged out every time we rotate SECRET_KEY, even though the old key is in SECRET_KEY_FALLBACKS. Why?" On Tuesday, in a new session, a small related edit. The first recording was not kind: | Monday | Claude Code | + knock v1 | |---|---|---| | Tool calls | 10 | 25 | | Cost | $0.21 | $0.39 | knock made it worse. The transcripts showed three bugs. Commit messages were cut to their subject line, so the one commit that explained the cause was useless. Search results filled up with off-topic notes. And code could only be found by name. I fixed all three and re-recorded, 3 runs per side this time. Monday: $0.28 without knock, $0.27 with it. A wash. Tuesday: knock was $0.03 more expensive. Attempt 2: a code map. It also lost. Notes felt like the wrong shape. An agent that's about to change code wants the code's structure, not someone's summary of it. So version 2 replaced notes with structure. One map call takes the task and returns a skeleton of the code that matters: function signatures, exact line ranges, what each function calls and who calls it, plus the commits that explain why the code changed. Then open reads exact code, and remember saves findings. On Flask: $0.25 without knock, $0.29 with it on Monday. The same on Tuesday. No win again. This time the transcripts made the reason obvious: map and open replaced Claude's own grep and Read almost one for one. Flask is about 200 files, and the model already knows it well. Finding the code was never the expensive part, so a better way to find code couldn't save anything. Claude even re-checked the fix commit with git show in every Monday session, though the map had already quoted that commit's explanation. The test that mattered: a big, unfamiliar codebase So I moved to Saleor, an e-commerce platform with 4,776 files, in a version released days before the test. The question spans a channel model, two Celery tasks, a custom beat schedule, settings and GraphQL validation: > "In one of our channels, fully paid checkouts never turn into orders automatically, even though automatic completion is enabled. Some of them are a few weeks old. What decides which checkouts get completed automatically, and where is that code?" This is where the money went. Here are two of the recorded runs, side by side: !The same Saleor investigation side by side. Claude Code alone: 30 tool calls, 3 min 22 s of model time, $0.49. With knock: 13 tool calls, 1 min 16 s, $0.25. Both answers correct. Without a map, Claude Code often hands the search to a sub-agent, which then explores on its own: grep, read, grep again. One of those sub-agents made 29 tool calls to answer a single question. With knock, map pointed straight at the task, the settings and the scheduler, and Claude confirmed the details with open instead of exploring. Finding 1: you pay per step, not per token Every step an agent takes sends the whole conversation back to the model. Prompt caching makes the repeated part cheaper, but not free, and the conversation keeps growing. So a 30-step investigation doesn't cost 30 small amounts. It costs 30 increasingly large ones. That's why knock is built to cut steps, not just shrink what each step returns. Without knock, the most expensive runs were the ones that used a sub-agent. With knock, none did. Finding 2: one pair of runs can lie After I posted the first results, someone on X pointed out that agent runs on the same question can swing a lot, so a single pair can flatter either side. They were right. So I recorded 5 more runs per side and pooled them with the first 3: !Cost per run for the same Saleor investigation, 8 runs per side. Without knock: median $0.42, ranging from $0.22 to $0.65, with 5 runs that delegated to a sub-agent. With knock: median $0.25, ranging from $0.16 to $0.38, with no sub-agent runs. Two things show up: - The spread halves. Without knock, runs split into two kinds: Claude either delegated to a sub-agent (23 to 37 tool calls, $0.39 to $0.65) or searched directly (8 to 20 calls). With knock it never delegated, and costs stayed between $0.16 and $0.38. - It's not a win every time. The cheapest run without knock ($0.22) beat knock's median. If I'd only recorded that one pair, I'd have reported no difference. With 3 runs, the saving looked like 49%. With 8, it's 40%. That's the number I trust. Finding 3: the tool has to know when to step aside knock also lost a test, and it kept losing it. The day after the Saleor investigation, a new session got a small edit: log how many checkouts the background job queued. Without knock, one regex search found the file: 3 tool calls, about $0.08. With knock it cost $0.14. That gap is structural. Claude Code has to read a file with its own Read tool before it can edit it, and loading an MCP server's tools takes a step too. So an easy edit with knock takes at least 4 steps where 3 would do. I narrowed the gap with an "edit mode" that returns the exact lines to read, but couldn't close it. The real fix was guidance. knock now tells the agent: use me to explore; for a small edit you can find with one grep, skip me. In the next batch, Claude followed it. It skipped knock for all three small Flask edits, and the whole two-day Saleor workflow (investigate, then fix) came out 39% cheaper ($0.70 → $0.43). On small Flask the two-day workflow was only 14% cheaper, and the ranges overlap, so I don't count that as a win. knock pays off when exploring is expensive: big or unfamiliar codebases. How knock works - Indexes locally, with zero AI calls. Saleor took 76 seconds the first time, and a refresh with no changes takes 0.3 s. Nothing leaves your machine. - Stays fresh. Before each answer it checks which files changed (about 35 ms on Saleor) and re-indexes them first. If more than 50 changed, it says the map is stale instead. - map returns the functions that matter for a task, their exact line ranges, who calls what, and why the code changed (from git history). - remember saves what the agent learned for the next session, and the note is flagged stale automatically when the code it cites changes. It's one command to add it to Claude Code (it needs uv): claude mcp add knock -- uvx knock-mcp serve --repo . --toolset map Caveats - One model (Claude Sonnet 5 in Claude Code), two repositories, and a handful of scenarios. 3 to 8 runs per condition is still a small sample. - The Saleor scenario isn't untouched: I fixed a ranking bug while checking knock's answers for these exact tasks, before recording. - Costs and times are as reported by Claude Code. Tested on macOS and Linux, not yet on Windows. Every recorded result, including the losses, is in the benchmark write-up. Try it knock is free and open source (Apache-2.0): HasanGilani/knock. It's on PyPI as knock-mcp and in the official MCP Registry. If you try it on a big codebase, I'd love to hear what you see, especially where it doesn't help. That's been the most useful data so far.
Still paying for Shopify, BigCommerce or WooCommerce? Saleor does the same job for $0. Django based open-sourced e-commerce storefront. ([Demo](demo.saleor.io), [Source Code](saleor/saleor))…