Two copies of a 31.9 MB tokenizer were most of my Rust proxy's resting memory
I've been porting CLIProxyAPI, a Go proxy that sits between coding CLIs (Codex CLI, Claude Code) and model providers, to Rust. It reads the same config.yaml and serves the same routes, and I mostly use it to put API-key providers and Chat Completions-only gateways behind one local endpoint. Repo: vayungodara/cliproxy-rs
On 0.2.0, my server's resting RSS rose from 26 MB to 89 MB over 7 hours of Claude and Codex traffic with 1 to 2 MB prompts. I reproduced the load in a 60-minute soak and ran heaptrack on it.
67.1 MB was still allocated at exit, and 63.8 MB of that was two tiktoken-rs o200k_base encoders, 31.9 MB each. The Claude count_tokens path built one and the Codex count_tokens path built the other. One encoder breaks down like this:
- encoder map: 9.6 MB
- decoder map: 9.6 MB, which token counting never uses
- sorted token list: 5.9 MB
- regexes: 6.8 MB, 5.9 MB of it 128 per-thread copies of the main regex
About 600,000 short token allocations each land on malloc's 32-byte minimum chunk, so one encoder is about 47 MB resident.
Five modules each kept their own lazily built CoreBPE static, seven in total (five o200k_base, two cl100k_base). Running every count path once built 164.1 MB of tokenizers. With one shared lazy encoder per encoding, the same sequence builds 71.0 MB. Go builds the same tokenizers in the same places, so the encoders have to exist; the fix was sharing them.
I checked fragmentation first: malloc_trim(0) through gdb changed RssAnon by 40 kB, so the floor was live data.
In the 60-minute soak, resting RSS went from 128-145 MB on 0.2.0 to 81-98 MB with the fix, and the peak from 220 to 169 MB. On my server, median RSS went from 86 to 61 MB after the fix shipped in 0.2.2.
The next round, in 0.2.4, was peak memory under large Claude requests. Ten requests waiting for their first token each held up to four copies of the body: one kept for a possible error log, the client's body kept for retries, a translated body nothing needed once the request was built, and the body the HTTP client keeps in case of a redirect. Dropping two of those and letting rewrites that change nothing pass the body through took the soak's peak RSS from 171-174 MB to 111-117 MB. Each waiting request still holds two copies, as the Go version does.
Last week I ran 0.4.0 and the Go v8.0.35 release binary side by side for an hour of streamed Claude and Codex requests of 200 KB to 2 MB, each on its own 4-vCPU GitHub runner. The Rust one peaked at 161.5 MB and Go at 198.3 MB, and on the same CPU model it used 49 ms of CPU per request against Go's 146 ms. After 5 minutes idle, though, Go had given more back: 75.8 MB against my 97.8 MB, because the shared o200k_base encoder, still the biggest thing in my idle floor, stays resident once count_tokens has built it.
It isn't faster than Go everywhere either. In the 0.1.0 benchmark (2-vCPU VM, fake upstream), Go handled 1,568 non-streaming requests/s against 1,168.
For parity, tests compare output byte for byte against fixtures recorded from the Go server, and a harness runs both proxies against the same scripted upstream. Parity is still incomplete: the audit against v8.0.35 has 1,951 items, 1,053 covered, 872 partial and 26 missing.
Setups and raw results for all of the above are in docs/BENCHMARKS.md: github.com/vayungodara/cliproxy-rs/blob/master/do…
2 comments
Panel memory only. AutoModpack syncs approved files. Keep backups.
spawnpoint.vayun.net
vayungodara/spawnpoint
4 views
I made a server panel that installs from Modrinth and tests the new pack before my friends get it (alpha)
Spawnpoint is the panel I use for our modded server. You search Modrinth (CurseForge too, with an API key), install, and it picks versions that fit your Minecraft, loader and Java, pulls in dependencies, and checks every download against its hash and size.
Before the change goes live it boots a copy of the server and has a real client join a throwaway one. Friends then get those exact files through AutoModpack.
Mod ID lookup on Modrinth isn't complete, so sometimes Spawnpoint finds a pack that works but can't prove it's the smallest change. The review sheet says so when that happens.
On my server the panel is light: health OK in 0.56 s after a start, about 60 MB with the game servers stopped (panel only).
Free, MIT, alpha.
Source and screenshots: vayungodara/spawnpoint
Tour: spawnpoint.vayun.net
**AI disclosure:** some code is AI-written. Agents did some of the coding (Claude via Claude Code, OpenAI GPT models via Amp, ampcode.com, an AI coding tool, with GPT-6 Astra reviewing risky changes) while I made the product and design decisions and tested on real Fabric, Forge and NeoForge servers. [Vayun: August to late-October models]
1 point
Spawnpoint: my self-hosted Minecraft panel for a server with friends (alpha)
I made Spawnpoint for our Minecraft server. I wanted the panel to use little memory and keep the everyday screens simple.
The tour shows the interface: spawnpoint.vayun.net
On my Linux server the panel service measured 54 to 67 MB in two readings with all game servers stopped. Test jobs and Minecraft itself need extra memory.
It runs servers itself or alongside Crafty. Mod changes get a test copy and one review sheet before approval, then AutoModpack syncs the approved files. Undo restores pack files and configs; world restore is separate.
It's free, MIT-licensed and alpha. Keep it on your LAN or VPN, and keep backups.
Code and setup: vayungodara/spawnpoint
1 point · 1 comment
Spawnpoint - a Minecraft panel I made for my friends' server (alpha)
This is Spawnpoint, my Minecraft panel. The video shows it better than I can describe it: spawnpoint.vayun.net
I wanted it to be quick to open and easy to understand. On my Linux server, one start reached health OK in 0.56 seconds. The panel used about 60 MB with the game servers stopped, not counting Minecraft itself.
Mod changes get tested on a copy. One review sheet shows what changed, the results and anything it couldn't check. AutoModpack gives friends the approved files, and there's pack undo.
It's free, open source and alpha. Keep backups. I'd like to hear whether the review screen makes sense the first time you see it.
Code: vayungodara/spawnpoint
1 point · 2 comments
@orcdev @theo you should checkout vayungodara/cliproxy-rs
11 views
How I check that my Rust port still does what the Go original does
cliproxy-rs is my Rust port of CLIProxyAPI, a Go proxy that puts Claude, Codex, Gemini and OpenAI-compatible APIs behind one local endpoint. The promise is that a Go user can point the Rust binary at their existing config.yaml and credential files and get the same routes and the same answers.
Trying requests by hand only tells you the requests you tried work. I wanted a coverage number I could trace back to evidence, item by item.
The checklist comes from the Go tree
A list I wrote by hand would only contain the things I remembered, so the checklist is extracted from the Go source at a pinned commit. Each item is one contract: a Go test file, a config key, a public or management route, a plugin RPC method, a CLI flag. At Go v8.0.35 that's 1,951 items. The biggest groups:
| Kind | Items |
|---|---:|
| Go test-suite files | 989 |
| Config leaves | 436 |
| Management routes | 186 |
| Public routes | 77 |
| Plugin RPC methods | 62 |
| CLI flags | 34 |
Every item gets one of three statuses:
- covered: implemented, and a Rust test or a fixture recorded from Go exercises it
- partial: implemented in part, or implemented with no test that pins Go's behaviour
- missing: not there
A Python script (audit.py) fills in most of it. Fixtures recorded from Go carry the Go test name and line, so it can match each Go test against the names cited in Rust tests and fixtures. A second script starts the server and records which method and path pairs actually route. When the scan gets something wrong I override it in a TSV, one row per item, with the evidence and a note.
The first audit was too generous
When I redid the audit for v0.2.6, the generated numbers were 854 covered, 722 partial, 111 missing. The README said 835 / 707 / 145, so it was already stale.
I went through all 833 items that weren't covered and 164 changed status. Most moved up (73 partial to covered, 36 missing to covered, 42 missing to partial). 13 went from covered down to partial, and those were the useful ones.
The scanner had counted a route as covered when a fixture hit the same path, even with a different HTTP method, so a GET test was vouching for a POST handler. Now the evidence has to match the method and the complete route.
Two more rules came out of that pass. A subsystem that works doesn't make a whole Go test suite covered: Go has 23 Claude alias-restoration cases, I replay 20, so that item is partial. And a Go test name in a Rust comment is a search hint. It doesn't prove the assertions were ported.
Moving the target
Go keeps shipping. I'd been comparing against v8.0.10, and moving to v8.0.35 meant going through 288 upstream commits and adding 264 checklist items. I kept all 1,687 existing IDs and appended the new ones at the end of each section, so branches I had in flight still point at the right item. Tests Go deleted are marked historical instead of disappearing.
The last audit (20 Oct, against v8.0.35) reads 1,053 covered, 872 partial and 26 missing.
Partial is the biggest bucket. It holds everything implemented in part or implemented without a test that pins Go's behaviour, and the audit won't round any of it up to covered.
If you're porting something
Generate the checklist from the original's source. Make "covered" need evidence a script can find, and make every manual override say why. My first honest pass moved 13 items backwards.
The checklist, the scripts and the per-item notes are in the repo under docs/parity-audit: vayungodara/cliproxy-rs
2 comments
Using your OpenCode Go plan's DeepSeek models in Codex CLI
OpenCode Go lists Codex as a validated client, but Codex can only use the Go models served on /responses. DeepSeek, GLM, Kimi and most of the others are Chat Completions only, and current Codex refuses wire_api = "chat".
I got around it with a local proxy that translates between the two. It's cliproxy-rs, which I maintain (free, MIT, written in Rust). Codex talks Responses to it on localhost, and it calls opencode.ai with your Go key.
The part of the proxy config that matters:
api-keys:
openai-compatibility:
- name: opencode-go
base-url: https://opencode.ai/zen/go/v1
headers:
x-opencode-session: $CPA-SESSION-ID
keys:
- api-key: <your Go key>
models:
- name: deepseek-v4.1-flash
alias: deepseek
You need the header line. Go answers 400 MissingSessionID without it, and $CPA-SESSION-ID gives each Codex conversation its own session ID, so Go can still route and cache per conversation.
I've run Codex 0.160 on deepseek-v4.1-flash this way: it read a file, patched it and ran a shell command. Other Chat Completions models on Go should work by adding them under models the same way, but I haven't tried GLM or Kimi through it yet. Recording of the DeepSeek run: github.com/vayungodara/cliproxy-rs/blob/master/do…
Repo: vayungodara/cliproxy-rs
3 points
Two 32 MB tokenizers: hunting a memory floor in a Rust proxy
cliproxy-rs is a Rust rewrite of CLIProxyAPI. It's one local server that Codex CLI, Claude Code and other tools talk to, and it forwards to whatever you've configured: your own API keys, your own accounts. I run it on a small Linux box under real coding-agent traffic.
On 0.2.0 that box showed something I didn't like. The resting memory, meaning what the process holds between requests, rose from 26 MB to 89 MB over about seven hours. Peaks were 300 to 400 MB. Nothing was failing, but a floor that only goes up is how a leak usually starts.
Reproducing it
My soak test only replayed Claude streams of 100 to 500 KB. The real traffic was different: Claude and Codex requests of 1 to 2 MB from long agent sessions, plus count_tokens calls. So I wrote a "field mix" replay with exactly that shape (four sessions, bodies averaging 1 MB) and ran 0.2.0 and the fix branch side by side on separate GitHub runners for an hour.
0.2.0 rested at 128 to 145 MB. That was clearly too much for a proxy that does no caching of its own.
What heaptrack found
After two batches of 100 field-mix turns, heaptrack showed 67.1 MB still allocated at exit. 63.8 MB of that was two o200k_base tokenizer encoders, 31.9 MB each. They were identical. Claude count_tokens had built one, and Codex count_tokens had built the other.
Five modules each kept their own lazily built tiktoken encoder in a static. Each was built the first time something needed a token count and never freed. On a server that counts tokens for both Claude and Codex models and streams Claude-format requests to Codex models, up to seven copies could exist, 164 MB of tables in total.
One encoder is bigger than its 31.9 MB of live data suggests. It holds about 600,000 short token allocations, and at malloc's 32-byte minimum chunk that comes to 47 MB resident.
I also checked whether glibc was just holding freed memory. It wasn't: after a trim, calling malloc_trim(0) again changed resident memory by 40 kB. The floor was live data.
The fix
One shared encoder per encoding, still built only when a count first needs it. The same mix now holds 71 MB of tables instead of 164.
Same hour of traffic, same runners:
| | 0.2.0 | 0.2.2 |
| --- | --- | --- |
| Resting RSS | 128.2 to 145.4 MB | 80.6 to 97.6 MB |
| Peak RSS (VmHWM) | 220.1 MB | 169.0 MB |
CPU per request didn't change (43.1 ms against 42.9 and 44.1 ms on one 2-vCPU machine).
What I haven't solved
- The replay steps up once and then stays flat. It never shows the slow climb from the field. My best guess is that the field server built its encoders hours apart, which would look like a slow rise, but I don't know that yet. A 24-hour trace of 0.2.2 is running now.
- Peaks are still big. A Codex request peaks at 8.8 times its body size, because request shaping rewrites the body pass by pass and each pass is a new copy. Claude requests peak at 6.0x.
- A fixed MALLOC_MMAP_THRESHOLD_ took 26.6 to 33.8 MB off the peak, but cost 16 to 31% more CPU per request, so I didn't ship it.
- The Go original is still faster on small requests. In my 0.1.0 benchmark, it served 1,568 non-streamed chat requests a second to cliproxy-rs's 1,168.
Keeping it fixed
Every release tag now runs an hour of the field mix, and a 5-hour run happens every Saturday. A run fails if the resting floor rises, if the peak goes over its bound or if any request fails. Heap budgets in the tests now cover the Codex and count_tokens routes as well as Claude.
The method and the raw rows are in docs/BENCHMARKS.md, and the code is at vayungodara/cliproxy-rs.
Built with some help from AI.
Mentions
Two copies of a 31.9 MB tokenizer were most of my Rust proxy's resting memory I've been porting CLIProxyAPI, a Go proxy that sits between coding CLIs (Codex CLI, Claude Code) and model providers, to Rust. It reads the same config.yaml and serves the same routes, and I mostly use it to put API-key providers and Chat Completions-only gateways behind one local endpoint. Repo: vayungodara/cliproxy-rs On 0.2.0, my server's resting RSS rose from 26 MB to 89 MB over 7 hours of Claude and Codex traffic with 1 to 2 MB prompts. I reproduced the load in a 60-minute soak and ran heaptrack on it. 67.1 MB was still allocated at exit, and 63.8 MB of that was two tiktoken-rs o200k_base encoders, 31.9 MB each. The Claude count_tokens path built one and the Codex count_tokens path built the other. One encoder breaks down like this: - encoder map: 9.6 MB - decoder map: 9.6 MB, which token counting never uses - sorted token list: 5.9 MB - regexes: 6.8 MB, 5.9 MB of it 128 per-thread copies of the main regex About 600,000 short token allocations each land on malloc's 32-byte minimum chunk, so one encoder is about 47 MB resident. Five modules each kept their own lazily built CoreBPE static, seven in total (five o200k_base, two cl100k_base). Running every count path once built 164.1 MB of tokenizers. With one shared lazy encoder per encoding, the same sequence builds 71.0 MB. Go builds the same tokenizers in the same places, so the encoders have to exist; the fix was sharing them. I checked fragmentation first: malloc_trim(0) through gdb changed RssAnon by 40 kB, so the floor was live data. In the 60-minute soak, resting RSS went from 128-145 MB on 0.2.0 to 81-98 MB with the fix, and the peak from 220 to 169 MB. On my server, median RSS went from 86 to 61 MB after the fix shipped in 0.2.2. The next round, in 0.2.4, was peak memory under large Claude requests. Ten requests waiting for their first token each held up to four copies of the body: one kept for a possible error log, the client's body kept for retries, a translated body nothing needed once the request was built, and the body the HTTP client keeps in case of a redirect. Dropping two of those and letting rewrites that change nothing pass the body through took the soak's peak RSS from 171-174 MB to 111-117 MB. Each waiting request still holds two copies, as the Go version does. Last week I ran 0.4.0 and the Go v8.0.35 release binary side by side for an hour of streamed Claude and Codex requests of 200 KB to 2 MB, each on its own 4-vCPU GitHub runner. The Rust one peaked at 161.5 MB and Go at 198.3 MB, and on the same CPU model it used 49 ms of CPU per request against Go's 146 ms. After 5 minutes idle, though, Go had given more back: 75.8 MB against my 97.8 MB, because the shared o200k_base encoder, still the biggest thing in my idle floor, stays resident once count_tokens has built it. It isn't faster than Go everywhere either. In the 0.1.0 benchmark (2-vCPU VM, fake upstream), Go handled 1,568 non-streaming requests/s against 1,168. For parity, tests compare output byte for byte against fixtures recorded from the Go server, and a harness runs both proxies against the same scripted upstream. Parity is still incomplete: the audit against v8.0.35 has 1,951 items, 1,053 covered, 872 partial and 26 missing. Setups and raw results for all of the above are in docs/BENCHMARKS.md: github.com/vayungodara/cliproxy-rs/blob/master/do…
Panel memory only. AutoModpack syncs approved files. Keep backups. spawnpoint.vayun.net vayungodara/spawnpoint
I made a server panel that installs from Modrinth and tests the new pack before my friends get it (alpha) Spawnpoint is the panel I use for our modded server. You search Modrinth (CurseForge too, with an API key), install, and it picks versions that fit your Minecraft, loader and Java, pulls in dependencies, and checks every download against its hash and size. Before the change goes live it boots a copy of the server and has a real client join a throwaway one. Friends then get those exact files through AutoModpack. Mod ID lookup on Modrinth isn't complete, so sometimes Spawnpoint finds a pack that works but can't prove it's the smallest change. The review sheet says so when that happens. On my server the panel is light: health OK in 0.56 s after a start, about 60 MB with the game servers stopped (panel only). Free, MIT, alpha. Source and screenshots: vayungodara/spawnpoint Tour: spawnpoint.vayun.net **AI disclosure:** some code is AI-written. Agents did some of the coding (Claude via Claude Code, OpenAI GPT models via Amp, ampcode.com, an AI coding tool, with GPT-6 Astra reviewing risky changes) while I made the product and design decisions and tested on real Fabric, Forge and NeoForge servers. [Vayun: August to late-October models]
Spawnpoint: my self-hosted Minecraft panel for a server with friends (alpha) I made Spawnpoint for our Minecraft server. I wanted the panel to use little memory and keep the everyday screens simple. The tour shows the interface: spawnpoint.vayun.net On my Linux server the panel service measured 54 to 67 MB in two readings with all game servers stopped. Test jobs and Minecraft itself need extra memory. It runs servers itself or alongside Crafty. Mod changes get a test copy and one review sheet before approval, then AutoModpack syncs the approved files. Undo restores pack files and configs; world restore is separate. It's free, MIT-licensed and alpha. Keep it on your LAN or VPN, and keep backups. Code and setup: vayungodara/spawnpoint
Spawnpoint - a Minecraft panel I made for my friends' server (alpha) This is Spawnpoint, my Minecraft panel. The video shows it better than I can describe it: spawnpoint.vayun.net I wanted it to be quick to open and easy to understand. On my Linux server, one start reached health OK in 0.56 seconds. The panel used about 60 MB with the game servers stopped, not counting Minecraft itself. Mod changes get tested on a copy. One review sheet shows what changed, the results and anything it couldn't check. AutoModpack gives friends the approved files, and there's pack undo. It's free, open source and alpha. Keep backups. I'd like to hear whether the review screen makes sense the first time you see it. Code: vayungodara/spawnpoint
@orcdev @theo you should checkout vayungodara/cliproxy-rs
How I check that my Rust port still does what the Go original does cliproxy-rs is my Rust port of CLIProxyAPI, a Go proxy that puts Claude, Codex, Gemini and OpenAI-compatible APIs behind one local endpoint. The promise is that a Go user can point the Rust binary at their existing config.yaml and credential files and get the same routes and the same answers. Trying requests by hand only tells you the requests you tried work. I wanted a coverage number I could trace back to evidence, item by item. The checklist comes from the Go tree A list I wrote by hand would only contain the things I remembered, so the checklist is extracted from the Go source at a pinned commit. Each item is one contract: a Go test file, a config key, a public or management route, a plugin RPC method, a CLI flag. At Go v8.0.35 that's 1,951 items. The biggest groups: | Kind | Items | |---|---:| | Go test-suite files | 989 | | Config leaves | 436 | | Management routes | 186 | | Public routes | 77 | | Plugin RPC methods | 62 | | CLI flags | 34 | Every item gets one of three statuses: - covered: implemented, and a Rust test or a fixture recorded from Go exercises it - partial: implemented in part, or implemented with no test that pins Go's behaviour - missing: not there A Python script (audit.py) fills in most of it. Fixtures recorded from Go carry the Go test name and line, so it can match each Go test against the names cited in Rust tests and fixtures. A second script starts the server and records which method and path pairs actually route. When the scan gets something wrong I override it in a TSV, one row per item, with the evidence and a note. The first audit was too generous When I redid the audit for v0.2.6, the generated numbers were 854 covered, 722 partial, 111 missing. The README said 835 / 707 / 145, so it was already stale. I went through all 833 items that weren't covered and 164 changed status. Most moved up (73 partial to covered, 36 missing to covered, 42 missing to partial). 13 went from covered down to partial, and those were the useful ones. The scanner had counted a route as covered when a fixture hit the same path, even with a different HTTP method, so a GET test was vouching for a POST handler. Now the evidence has to match the method and the complete route. Two more rules came out of that pass. A subsystem that works doesn't make a whole Go test suite covered: Go has 23 Claude alias-restoration cases, I replay 20, so that item is partial. And a Go test name in a Rust comment is a search hint. It doesn't prove the assertions were ported. Moving the target Go keeps shipping. I'd been comparing against v8.0.10, and moving to v8.0.35 meant going through 288 upstream commits and adding 264 checklist items. I kept all 1,687 existing IDs and appended the new ones at the end of each section, so branches I had in flight still point at the right item. Tests Go deleted are marked historical instead of disappearing. The last audit (20 Oct, against v8.0.35) reads 1,053 covered, 872 partial and 26 missing. Partial is the biggest bucket. It holds everything implemented in part or implemented without a test that pins Go's behaviour, and the audit won't round any of it up to covered. If you're porting something Generate the checklist from the original's source. Make "covered" need evidence a script can find, and make every manual override say why. My first honest pass moved 13 items backwards. The checklist, the scripts and the per-item notes are in the repo under docs/parity-audit: vayungodara/cliproxy-rs
Using your OpenCode Go plan's DeepSeek models in Codex CLI OpenCode Go lists Codex as a validated client, but Codex can only use the Go models served on /responses. DeepSeek, GLM, Kimi and most of the others are Chat Completions only, and current Codex refuses wire_api = "chat". I got around it with a local proxy that translates between the two. It's cliproxy-rs, which I maintain (free, MIT, written in Rust). Codex talks Responses to it on localhost, and it calls opencode.ai with your Go key. The part of the proxy config that matters: api-keys: openai-compatibility: - name: opencode-go base-url: https://opencode.ai/zen/go/v1 headers: x-opencode-session: $CPA-SESSION-ID keys: - api-key: <your Go key> models: - name: deepseek-v4.1-flash alias: deepseek You need the header line. Go answers 400 MissingSessionID without it, and $CPA-SESSION-ID gives each Codex conversation its own session ID, so Go can still route and cache per conversation. I've run Codex 0.160 on deepseek-v4.1-flash this way: it read a file, patched it and ran a shell command. Other Chat Completions models on Go should work by adding them under models the same way, but I haven't tried GLM or Kimi through it yet. Recording of the DeepSeek run: github.com/vayungodara/cliproxy-rs/blob/master/do… Repo: vayungodara/cliproxy-rs
Two 32 MB tokenizers: hunting a memory floor in a Rust proxy cliproxy-rs is a Rust rewrite of CLIProxyAPI. It's one local server that Codex CLI, Claude Code and other tools talk to, and it forwards to whatever you've configured: your own API keys, your own accounts. I run it on a small Linux box under real coding-agent traffic. On 0.2.0 that box showed something I didn't like. The resting memory, meaning what the process holds between requests, rose from 26 MB to 89 MB over about seven hours. Peaks were 300 to 400 MB. Nothing was failing, but a floor that only goes up is how a leak usually starts. Reproducing it My soak test only replayed Claude streams of 100 to 500 KB. The real traffic was different: Claude and Codex requests of 1 to 2 MB from long agent sessions, plus count_tokens calls. So I wrote a "field mix" replay with exactly that shape (four sessions, bodies averaging 1 MB) and ran 0.2.0 and the fix branch side by side on separate GitHub runners for an hour. 0.2.0 rested at 128 to 145 MB. That was clearly too much for a proxy that does no caching of its own. What heaptrack found After two batches of 100 field-mix turns, heaptrack showed 67.1 MB still allocated at exit. 63.8 MB of that was two o200k_base tokenizer encoders, 31.9 MB each. They were identical. Claude count_tokens had built one, and Codex count_tokens had built the other. Five modules each kept their own lazily built tiktoken encoder in a static. Each was built the first time something needed a token count and never freed. On a server that counts tokens for both Claude and Codex models and streams Claude-format requests to Codex models, up to seven copies could exist, 164 MB of tables in total. One encoder is bigger than its 31.9 MB of live data suggests. It holds about 600,000 short token allocations, and at malloc's 32-byte minimum chunk that comes to 47 MB resident. I also checked whether glibc was just holding freed memory. It wasn't: after a trim, calling malloc_trim(0) again changed resident memory by 40 kB. The floor was live data. The fix One shared encoder per encoding, still built only when a count first needs it. The same mix now holds 71 MB of tables instead of 164. Same hour of traffic, same runners: | | 0.2.0 | 0.2.2 | | --- | --- | --- | | Resting RSS | 128.2 to 145.4 MB | 80.6 to 97.6 MB | | Peak RSS (VmHWM) | 220.1 MB | 169.0 MB | CPU per request didn't change (43.1 ms against 42.9 and 44.1 ms on one 2-vCPU machine). What I haven't solved - The replay steps up once and then stays flat. It never shows the slow climb from the field. My best guess is that the field server built its encoders hours apart, which would look like a slow rise, but I don't know that yet. A 24-hour trace of 0.2.2 is running now. - Peaks are still big. A Codex request peaks at 8.8 times its body size, because request shaping rewrites the body pass by pass and each pass is a new copy. Claude requests peak at 6.0x. - A fixed MALLOC_MMAP_THRESHOLD_ took 26.6 to 33.8 MB off the peak, but cost 16 to 31% more CPU per request, so I didn't ship it. - The Go original is still faster on small requests. In my 0.1.0 benchmark, it served 1,568 non-streamed chat requests a second to cliproxy-rs's 1,168. Keeping it fixed Every release tag now runs an hour of the field mix, and a 5-hour run happens every Saturday. A run fails if the resting floor rises, if the peak goes over its bound or if any request fails. Heap budgets in the tests now cover the Codex and count_tokens routes as well as Claude. The method and the raw rows are in docs/BENCHMARKS.md, and the code is at vayungodara/cliproxy-rs. Built with some help from AI.