Idol Image-to-3D model is amazing! 🤩
So I decided to port it to Apple MLX on the fly!
There are some pieces still in MPS and more optimizations to bring in, step by step 💪
Thanks for this amazing model @speridlabs
Here it is: ivanfioravanti/idol-sdk-mlx t.co
40 likes · 6 reposts · 2,315 views
Scripts, raw outputs and the PASS/FAIL check: ivanfioravanti/Unlimited-OCR (bench/, branches benchmark-m3ultra-mlx and benchmark-gb10-findings)
3 likes · 1 repost · 413 views
@geldeki @coldniko I have the same box. I run IQ_S with 128k context in the and get roughly 100-130ts consistently even with full context. From the screenshot I see the context stays low? Did you try it when going over the 32k context and expert swaps ?
Did you try github.com/ivanfioravanti/llm_context_benchmarks/… ?
I run 115 TS and 3500 average prefill.
If I use open code I hit 150 ts decode constantly.
2 likes · 41 views
Been running @ivanfioravanti fork with DeepSeek V4.1 Flash 1017 and it's kinda wild. Very little prose if any. It's like the agent thinks only in terms of tool calls.
ivanfioravanti/ds4-metal
8 likes · 948 views
Video of DeepSeek Harness with DeepSeek V4.1 Flash 1017 running on DwarfStar - ds4 on a Single M3 Ultra 512GB after optimizations and fixes (image input not declared in /v1/models) from @kernelpool and myself.
My repo here: ivanfioravanti/ds4-metal
This is focused on Apple silicon only and per M5 specifically, it's the one I use in my "production".
Not bad and there is more room for improvement!
PR coming soon @antirez 🙌 probably from kernelpool branch.
25 likes · 1 repost · 1,592 views
Is it me or LM Studio cannot handle large context files?
I noticed some files were giving and error and not loading, others just loading 3 Lines...
So I tried to test it, dragged and dropped this text file for context benchmarks:
github.com/ivanfioravanti/llm_context_benchmarks/…
to a qwen3.8-flash-next with the context set to 262144 and enough memory.
And i get:
>Chosen context injection strategy: 'retrieval'. Retrieval is optimal for the size of content provided
No relevant citations found for user query
Done predicting
Then the model proceeds to "User asked me to summarize this file... but there is no file..." blah blah blah
Even using a smaller model, it seems LM Studio cannot chew large context files.
64k context txt file works though.
Anyone experienced this.
im using the default rag v1, with 10 threshold.
1 point · 7 comments
Mentions
Idol Image-to-3D model is amazing! 🤩 So I decided to port it to Apple MLX on the fly! There are some pieces still in MPS and more optimizations to bring in, step by step 💪 Thanks for this amazing model @speridlabs Here it is: ivanfioravanti/idol-sdk-mlx t.co
Scripts, raw outputs and the PASS/FAIL check: ivanfioravanti/Unlimited-OCR (bench/, branches benchmark-m3ultra-mlx and benchmark-gb10-findings)
@geldeki @coldniko I have the same box. I run IQ_S with 128k context in the and get roughly 100-130ts consistently even with full context. From the screenshot I see the context stays low? Did you try it when going over the 32k context and expert swaps ? Did you try github.com/ivanfioravanti/llm_context_benchmarks/… ? I run 115 TS and 3500 average prefill. If I use open code I hit 150 ts decode constantly.
Been running @ivanfioravanti fork with DeepSeek V4.1 Flash 1017 and it's kinda wild. Very little prose if any. It's like the agent thinks only in terms of tool calls. ivanfioravanti/ds4-metal
Video of DeepSeek Harness with DeepSeek V4.1 Flash 1017 running on DwarfStar - ds4 on a Single M3 Ultra 512GB after optimizations and fixes (image input not declared in /v1/models) from @kernelpool and myself. My repo here: ivanfioravanti/ds4-metal This is focused on Apple silicon only and per M5 specifically, it's the one I use in my "production". Not bad and there is more room for improvement! PR coming soon @antirez 🙌 probably from kernelpool branch.
Is it me or LM Studio cannot handle large context files? I noticed some files were giving and error and not loading, others just loading 3 Lines... So I tried to test it, dragged and dropped this text file for context benchmarks: github.com/ivanfioravanti/llm_context_benchmarks/… to a qwen3.8-flash-next with the context set to 262144 and enough memory. And i get: >Chosen context injection strategy: 'retrieval'. Retrieval is optimal for the size of content provided No relevant citations found for user query Done predicting Then the model proceeds to "User asked me to summarize this file... but there is no file..." blah blah blah Even using a smaller model, it seems LM Studio cannot chew large context files. 64k context txt file works though. Anyone experienced this. im using the default rag v1, with 10 threshold.