I've been using pi.dev for the last 5 days and I'm a little sad at how much better it is vs hermes
I've been a hardcore hermes user for the last 7 month and hermes has been a wonderful journey. I've learned a lot about the agentic world. I love being able to use which ever LLM model I want with hermes, but on a whim, I decided to try out pi.dev over the weekend.
the thing that impressed me about pi.dev is how fast, responsive and not bloaty it is.
I pointed pi to my hermes/agentic platform and within a few minutes, pi become my meta harness.
it builds skills as needed, it understands the lay of the land, has easy access to MCP servers and it's able to drive hermes with ease.
over the last 5 days, pi has been debloating and cleaning up all of the mess I created with hermes. Pi now drives claude, codex and hermes without much "training" from me.
agegr/pi-web is better than nesquena/hermes-webui and I think that's due to pi itself being "lighter"
anyway - just wanted to share my exp. I still think hermes is great, but try out pi if you want something lighter weight
225 points · 98 comments
My 5060ti 16GB setup (Qwen3.8 27b vs Qwen Flash Next)
TL;DR, pictures of horses, and conclusion below my evidence of neurodivergence.
I've spent a few weeks testing different setups in the evenings. It took a long time so I wanted to share my findings with the community to save time for others.
I am still learning. Please be kind and offer advice if you think I'm wrong.
LLM uses: Home assistant voice (STT>LLM>TTS), Pi agent coding, Open Web UI and a few other apps that use it (SparkyFitness, Mealie, etc.)
Hardware:
- 5060ti 16GB (used nvoc to +150 on core, +2,500 on memory, and raise power limit to 190w)
- Intel core ultra 5 225
- 110GB ram (128GB on the host, but this is in a VM) - 5,600MT/s g2 stable with 4 sticks but needed over-volt :'(
Setups tested:
I really like this Pi Web UI so I created a docker container to run that. I also have separate containers for my other services (like Open WebUI).
I use my CPU's NPU for STT, and CPU for TTS.
I tested different models:
- Gemma4:12b (multiple variants) <- These were all awful. Terrible at coding. Often encountered thinking loops. One time it exhausted 32k tokens reasonining on the prompt "write a 5 sentence joke". I was running Unsloth's Q8 and don't understand how people like this model.
- gemma-4-26B-A4B-it-UD-Q3_K_XL <- A lot better than 12b, but only for non-technical chat. It's very fast, but for the amount of ram it needs I expect better overall quality and accuracy.
- Bucoid/Qwen3.8-27B-Heretic-Ara-16GB-VRAM-IQ4-XS-MTP-GGUF <- Awful quality. Much worse than non-heretic. I wouldn't recommend using uncensored or heretic models.
- Bucoid/Qwen3.8-27B-IQ4-XS-MTP-16GB-VRAM-GGUF <- Performs really well. Fits entirely in VRAM (on its own) and with q8 kv cache I was able to fit around 80k context with MTP and 100+ without. Approx 25~30 tokens a second generation. Really good model.
- ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF (IQ_3S) <- Performed even better with context and tokens, but I did not compare quality to Bucoid's. It fits in VRAM with q8 kv cache and 120k~ context which is crazy. Approx 25~30 tokens a second generation. My recommended 27b model.
- ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF (IQ3_S) <- Running on Strata overflowing to system ram, I'm running a 260k~ context with 60~ tokens a second generation and it has found and fixed multiple mistakes from the 27b models. My recommended Flash-Next model.
I know there's a bit of dislike for Strata online, and the setup is very janky, but it gave me more context and performance compared to llama.cpp on its own.
Findings:
I trust benchmarks like I trust politicians, but I did leave a script that tests different llama.cpp variations + lm-eval benchmarks running for 3 days and found a few things. It's important to note benchmarking is slow, so I only used gsm8k, human-eval and MBPP.
1. I didn't try anything less than IQ3_S. Even IQ3_S feels low, I'd be more comfortable at Q4 but I only have one GPU.
2. Heretic, uncensored and abliterated models (I tested 5 or so different ones) always performed worse in the benchmarks. It's not worth it.
3. Q8 kv cache consistently produced better quality results over Q4, and it's worth the reduced context size to keep Q8 on these models. This was seen in lm-eval results and SVG prompts. Q4 was using 2x the amount of tokens to reason because it was more likely to get stuck in loops, so the additional context you get is not worth it. Additionally, on 200 gsm8k benchmarks Q4/Q4 scored 70% versus Q8/Q8 getting 73.5%. I also tried Q4/Q8 (used more tokens but got 75%) and Q8/Q4 (used less tokens but got 73%). Note: This is a very small sample size so there might be some normal noise/variation in these results.
4. Changing batch and ubatch values gives a slight increase in prompt processing times but rewards you with slightly more VRAM for context. This is definitely worth doing in my experience.
5. MTP wasn't worth it on the dense models, the higher context window was better since these models think a lot.
6. ASCII pruning will also cut a few hundred megabytes from the dense models. This further increases your context size. I saw no problems after testing pruned models so recommend it if you have 27b.
7. I had a script loop through llama.cpp combinations and run lm-eval benchmarks to compare performance and quality. For the 27b model, the winning command for me was:
Qwen3.8 27b vs Strata Flash Next (both iq3_s)
1. Non-coding: Both 27b and flash-next did well, but 27b gave better responses. The formatting was nicer and it used a better layout.
2. Coding: Flash-next did better for me. It found and fixed issues from 27b's coding. I sent samples to Gemini's new Argon model and it heavily preferred flash-next, finding significantly less problems. Much faster, too.
3. Generating SVGs: Weird benchmark that Reddit seems to like. 27b was a clear winner here. Images below.
4. Overall, Flash-Next captured more details than 27b, but didn't go into as much depth on each detail. A good agent harness with the right extensions, planning, and prompts can create a better overall output in the same amount of time and tokens (in my experience and use case) but requires significant system ram.
Paying my Reddit SVG tax:
27B
preview.redd.it
Flash-Next (neckmaxxed??)
Strata Flash-Next IQ3_s \"Generate an SVG of Horse sitting in an F1 car. Return one complete, self-contained SVG document (starting with <svg and ending with <\/svg>, with a viewBox) and nothing else\"
TL;DR:
- Do you have lots of system RAM? You might find Strata with Flash-Next IQ3_S is better than Qwen3.8 27B IQ3_S.
- Do you have 16GB VRAM or more? Stick with Q8 KV cache.
- Do you have lots of compute time available? Use both! Flash-Next to do the work (since its faster) and have 27b review and check.
- Don't have much system ram and your card is less than 16GB? The quality and performance is only just 'good enough' on 16GB. I'd pay for Claude's API while hoping second hand hardware prices come down soon, unless Qwen4 can deliver some miracles.
Do you have any feedback or suggestions for me? Please be kind
7 points · 10 comments
Mentions
@ljsabc 手机上不适合再用终端了，需要给手机做过适配的harness会比较好些。我这边用pi-web还不错，感兴趣可以试试agegr/pi-web
I've been using pi.dev for the last 5 days and I'm a little sad at how much better it is vs hermes I've been a hardcore hermes user for the last 7 month and hermes has been a wonderful journey. I've learned a lot about the agentic world. I love being able to use which ever LLM model I want with hermes, but on a whim, I decided to try out pi.dev over the weekend. the thing that impressed me about pi.dev is how fast, responsive and not bloaty it is. I pointed pi to my hermes/agentic platform and within a few minutes, pi become my meta harness. it builds skills as needed, it understands the lay of the land, has easy access to MCP servers and it's able to drive hermes with ease. over the last 5 days, pi has been debloating and cleaning up all of the mess I created with hermes. Pi now drives claude, codex and hermes without much "training" from me. agegr/pi-web is better than nesquena/hermes-webui and I think that's due to pi itself being "lighter" anyway - just wanted to share my exp. I still think hermes is great, but try out pi if you want something lighter weight
wow, agegr/pi-web is so great if you combine it with tailscale.
My 5060ti 16GB setup (Qwen3.8 27b vs Qwen Flash Next) TL;DR, pictures of horses, and conclusion below my evidence of neurodivergence. I've spent a few weeks testing different setups in the evenings. It took a long time so I wanted to share my findings with the community to save time for others. I am still learning. Please be kind and offer advice if you think I'm wrong. LLM uses: Home assistant voice (STT>LLM>TTS), Pi agent coding, Open Web UI and a few other apps that use it (SparkyFitness, Mealie, etc.) Hardware: - 5060ti 16GB (used nvoc to +150 on core, +2,500 on memory, and raise power limit to 190w) - Intel core ultra 5 225 - 110GB ram (128GB on the host, but this is in a VM) - 5,600MT/s g2 stable with 4 sticks but needed over-volt :'( Setups tested: I really like this Pi Web UI so I created a docker container to run that. I also have separate containers for my other services (like Open WebUI). I use my CPU's NPU for STT, and CPU for TTS. I tested different models: - Gemma4:12b (multiple variants) <- These were all awful. Terrible at coding. Often encountered thinking loops. One time it exhausted 32k tokens reasonining on the prompt "write a 5 sentence joke". I was running Unsloth's Q8 and don't understand how people like this model. - gemma-4-26B-A4B-it-UD-Q3_K_XL <- A lot better than 12b, but only for non-technical chat. It's very fast, but for the amount of ram it needs I expect better overall quality and accuracy. - Bucoid/Qwen3.8-27B-Heretic-Ara-16GB-VRAM-IQ4-XS-MTP-GGUF <- Awful quality. Much worse than non-heretic. I wouldn't recommend using uncensored or heretic models. - Bucoid/Qwen3.8-27B-IQ4-XS-MTP-16GB-VRAM-GGUF <- Performs really well. Fits entirely in VRAM (on its own) and with q8 kv cache I was able to fit around 80k context with MTP and 100+ without. Approx 25~30 tokens a second generation. Really good model. - ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF (IQ_3S) <- Performed even better with context and tokens, but I did not compare quality to Bucoid's. It fits in VRAM with q8 kv cache and 120k~ context which is crazy. Approx 25~30 tokens a second generation. My recommended 27b model. - ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF (IQ3_S) <- Running on Strata overflowing to system ram, I'm running a 260k~ context with 60~ tokens a second generation and it has found and fixed multiple mistakes from the 27b models. My recommended Flash-Next model. I know there's a bit of dislike for Strata online, and the setup is very janky, but it gave me more context and performance compared to llama.cpp on its own. Findings: I trust benchmarks like I trust politicians, but I did leave a script that tests different llama.cpp variations + lm-eval benchmarks running for 3 days and found a few things. It's important to note benchmarking is slow, so I only used gsm8k, human-eval and MBPP. 1. I didn't try anything less than IQ3_S. Even IQ3_S feels low, I'd be more comfortable at Q4 but I only have one GPU. 2. Heretic, uncensored and abliterated models (I tested 5 or so different ones) always performed worse in the benchmarks. It's not worth it. 3. Q8 kv cache consistently produced better quality results over Q4, and it's worth the reduced context size to keep Q8 on these models. This was seen in lm-eval results and SVG prompts. Q4 was using 2x the amount of tokens to reason because it was more likely to get stuck in loops, so the additional context you get is not worth it. Additionally, on 200 gsm8k benchmarks Q4/Q4 scored 70% versus Q8/Q8 getting 73.5%. I also tried Q4/Q8 (used more tokens but got 75%) and Q8/Q4 (used less tokens but got 73%). Note: This is a very small sample size so there might be some normal noise/variation in these results. 4. Changing batch and ubatch values gives a slight increase in prompt processing times but rewards you with slightly more VRAM for context. This is definitely worth doing in my experience. 5. MTP wasn't worth it on the dense models, the higher context window was better since these models think a lot. 6. ASCII pruning will also cut a few hundred megabytes from the dense models. This further increases your context size. I saw no problems after testing pruned models so recommend it if you have 27b. 7. I had a script loop through llama.cpp combinations and run lm-eval benchmarks to compare performance and quality. For the 27b model, the winning command for me was: Qwen3.8 27b vs Strata Flash Next (both iq3_s) 1. Non-coding: Both 27b and flash-next did well, but 27b gave better responses. The formatting was nicer and it used a better layout. 2. Coding: Flash-next did better for me. It found and fixed issues from 27b's coding. I sent samples to Gemini's new Argon model and it heavily preferred flash-next, finding significantly less problems. Much faster, too. 3. Generating SVGs: Weird benchmark that Reddit seems to like. 27b was a clear winner here. Images below. 4. Overall, Flash-Next captured more details than 27b, but didn't go into as much depth on each detail. A good agent harness with the right extensions, planning, and prompts can create a better overall output in the same amount of time and tokens (in my experience and use case) but requires significant system ram. Paying my Reddit SVG tax: 27B preview.redd.it Flash-Next (neckmaxxed??) Strata Flash-Next IQ3_s \"Generate an SVG of Horse sitting in an F1 car. Return one complete, self-contained SVG document (starting with <svg and ending with <\/svg>, with a viewBox) and nothing else\" TL;DR: - Do you have lots of system RAM? You might find Strata with Flash-Next IQ3_S is better than Qwen3.8 27B IQ3_S. - Do you have 16GB VRAM or more? Stick with Q8 KV cache. - Do you have lots of compute time available? Use both! Flash-Next to do the work (since its faster) and have 27b review and check. - Don't have much system ram and your card is less than 16GB? The quality and performance is only just 'good enough' on 16GB. I'd pay for Claude's API while hoping second hand hardware prices come down soon, unless Qwen4 can deliver some miracles. Do you have any feedback or suggestions for me? Please be kind