simple-llm

I ran a simple experiment on Cloudy last week. It cut the kernels in my LLM inference library (SimpleLLM) from 5,762 lines to 297 and made it decode 11-15% faster than vLLM 0.32.0 (latest). (using a single prompt) Blog: cloudy.so Code: naklecha/simple-llm t.co

i replaced simple-llm's old external kernels (~5.8k lines) with a faster & simpler ~300 lines triton kernel file. optimizations: github.com/naklecha/simple-llm/blob/master/docs/o… t.co

@zack_overflow kinda my thesis too when i was building simple-llm (naklecha/simple-llm), these days i want almost everything i do to be in simple scripts.