@StasKlymenko I ran 3 iterations x 89 tasks x 9 harnesses. This took 12+ hours on 5-9 strong machines (Droplets). All individual scores: github.com/phuryn/mini-harness/tree/main/benchmar…
I agree many are close. Cost and time differences are more clear.
But rather than adding extra runs, I'm repeating the same for DeepSeek V4.1 Flash 1017. Next, maybe Luna.
Multiplying by various models seems to have more sense than pinning Haiku 5.5.
1 like · 125 views
Crazy. Full Terminal Bench 2.1, 89 tasks, 3 runs. My Mini Harness (~1,400 lines of Python) wins with Pi (155K), mini-swe-agent (4K), and OpenCode 2.
Open source: phuryn/mini-harness
Next, I'm running the same bench with DeepSeek. I'll be improving it further to support OpenAI endpoints and any model.
26 likes · 2 reposts · 2,371 views
@JosepiBuildsIt Hey, thanks. Just published all individual scores with harnesses, models, runs, and individual tasks: github.com/phuryn/mini-harness/tree/main/benchmar…
Terminal Bench 2.1 is here: github.com/phuryn/mini-harness/tree/main/benchmar…
1 like · 66 views
If that helps you learn and experiment, star the repo: phuryn/mini-harness
2 likes · 427 views
Everyone is discussing what a harness is. Nobody shows you how to build one. And everyone can.
So, I built my own Claude Code: phuryn/mini-harness
It's called Mini Harness: about 1,200 lines of Python.
Other minimal harnesses exist, but none has /goal mode, permissions + auto mode, skills and MCP servers. I wanted a fully working one to explain how a harness is built, down to Anthropic's server-side compaction.
Surprisingly, it became the top harness on Claude Haiku.
How it works: 🧵
73 likes · 3 reposts · 5,269 views
Mentions
@StasKlymenko I ran 3 iterations x 89 tasks x 9 harnesses. This took 12+ hours on 5-9 strong machines (Droplets). All individual scores: github.com/phuryn/mini-harness/tree/main/benchmar… I agree many are close. Cost and time differences are more clear. But rather than adding extra runs, I'm repeating the same for DeepSeek V4.1 Flash 1017. Next, maybe Luna. Multiplying by various models seems to have more sense than pinning Haiku 5.5.
Crazy. Full Terminal Bench 2.1, 89 tasks, 3 runs. My Mini Harness (~1,400 lines of Python) wins with Pi (155K), mini-swe-agent (4K), and OpenCode 2. Open source: phuryn/mini-harness Next, I'm running the same bench with DeepSeek. I'll be improving it further to support OpenAI endpoints and any model.
@JosepiBuildsIt Hey, thanks. Just published all individual scores with harnesses, models, runs, and individual tasks: github.com/phuryn/mini-harness/tree/main/benchmar… Terminal Bench 2.1 is here: github.com/phuryn/mini-harness/tree/main/benchmar…
If that helps you learn and experiment, star the repo: phuryn/mini-harness
Everyone is discussing what a harness is. Nobody shows you how to build one. And everyone can. So, I built my own Claude Code: phuryn/mini-harness It's called Mini Harness: about 1,200 lines of Python. Other minimal harnesses exist, but none has /goal mode, permissions + auto mode, skills and MCP servers. I wanted a fully working one to explain how a harness is built, down to Anthropic's server-side compaction. Surprisingly, it became the top harness on Claude Haiku. How it works: 🧵