SGLang: Fast inference for LLMs and multimodal models (37.1k stars)
preview.redd.it
What it does
All text below is quoted directly from the project README.
>SGLang is an open-source inference framework for large language, vision-language, and diffusion models, optimized for agentic workloads, large-scale serving, and RL rollouts. SGLang Diffusion is its built-in image and video generation engine, included in this repository and the sglang Python package.
Get Started:
>Pull the Docker image, which includes SGLang and its dependencies: docker pull lmsysorg/sglang:latest
>Alternatively, install SGLang in an activated Python environment with uv: uv pip install --prerelease=allow sglang
Supported Hardware:
>SGLang supports a wide range of GPUs, TPUs, NPUs, CPUs, and Apple Silicon platforms.
- NVIDIA - B200/B300/GB200/GB300; H100/H200/H800/H20; A100; RTX 30/40/50 series, RTX 6000 Ada / PRO 6000; DGX Spark, Jetson Orin
- AMD - Instinct MI300X, MI325X, MI350X, MI355X
- Google TPU - v6e, v7; SGL-JAX / SGL-torchtpu
- Intel (GPU / CPU) - Arc / Arc Pro B-Series GPUs, Xeon CPUs
- Apple Silicon - Macs via Metal / MLX
- Huawei Ascend - A2, A3, 950PR/DT NPUs
- Moore Threads - MTT S5000 GPUs
>Integrations in progress: AWS Trainium, Alibaba T-Head PPU, Cambricon MLU, Qualcomm QAIC, MetaX, Hygon HCU/DCU, Iluvatar CoreX, and more.
SGL Ecosystem:
- Diffusion - SGLang Diffusion - Built into SGLang for image and video generation with diffusion models.
- Audio - SGLang Omni - Audio model serving for text-to-speech (TTS) and automatic speech recognition (ASR).
- Speculative Decoding - SpecForge - Train draft models for speculative decoding and deploy them with SGLang.
- RL and Post-Training - Miles, slime, AReaL, Tunix, verl - Training frameworks that integrate SGLang for rollout generation.
- Deployment and Orchestration - SMG, RBG, llm-d, Ray Serve, NVIDIA Dynamo - Deploy and scale SGLang inference services with routing, load balancing, and cluster orchestration.
- Education - Mini-SGLang, zero-to-sglang, DeepLearning.AI course - Learn inference engine design and efficient text and image generation through code and hands-on courses.
Trusted by Industry and Research:
>SGLang serves production workloads across AI labs, cloud platforms, enterprises, and universities.
>SGLang is hosted by LMSYS, a non-profit open-source organization.
Repository
sgl-project/sglang
Repository stats
|Metric|Value|
|:-|:-|
|Stars|37,117|
|Forks|9,547|
|Last release|v0.5.21 (2026-10-02)|
|Languages|Python 83%, Rust 10%, Cuda 4.3%|
|Contributors|2,158|
|AI contributor accounts (Claude / Codex / Cursor / Copilot)|claude (1,166 commits), cursoragent (118), Copilot (6)|
|AI co-authored commits|Claude: 1,197, Cursor: 124, Copilot: 15, Codex: 10|
Main contributor
|Field|Value|
|:-|:-|
|Name|Lianmin Zheng|
|GitHub|@merrymercy|
|Commits to this repo|1,359 (top contributor)|
|Bio|Engineer|
|Company / Location|Meta / Bay Area, CA|
|Website|lmzheng.net|
|X / Twitter|@lm_zheng|
|Followers|4,199|
|Public repos|54|
|On GitHub since|2015-10-13|
--------------------------------------------------------------------------------------
Discover the best GitHub repos without the endless scrolling.
Get top repos organized by category (AI, DevOps, Databases & more), plus key takeaways from developers' most useful comments - all in one free, twice-weekly newsletter.
👉 Get the best repos & community insights - Free
3 points
Mentions
SGLang: Fast inference for LLMs and multimodal models (37.1k stars) preview.redd.it What it does All text below is quoted directly from the project README. >SGLang is an open-source inference framework for large language, vision-language, and diffusion models, optimized for agentic workloads, large-scale serving, and RL rollouts. SGLang Diffusion is its built-in image and video generation engine, included in this repository and the sglang Python package. Get Started: >Pull the Docker image, which includes SGLang and its dependencies: docker pull lmsysorg/sglang:latest >Alternatively, install SGLang in an activated Python environment with uv: uv pip install --prerelease=allow sglang Supported Hardware: >SGLang supports a wide range of GPUs, TPUs, NPUs, CPUs, and Apple Silicon platforms. - NVIDIA - B200/B300/GB200/GB300; H100/H200/H800/H20; A100; RTX 30/40/50 series, RTX 6000 Ada / PRO 6000; DGX Spark, Jetson Orin - AMD - Instinct MI300X, MI325X, MI350X, MI355X - Google TPU - v6e, v7; SGL-JAX / SGL-torchtpu - Intel (GPU / CPU) - Arc / Arc Pro B-Series GPUs, Xeon CPUs - Apple Silicon - Macs via Metal / MLX - Huawei Ascend - A2, A3, 950PR/DT NPUs - Moore Threads - MTT S5000 GPUs >Integrations in progress: AWS Trainium, Alibaba T-Head PPU, Cambricon MLU, Qualcomm QAIC, MetaX, Hygon HCU/DCU, Iluvatar CoreX, and more. SGL Ecosystem: - Diffusion - SGLang Diffusion - Built into SGLang for image and video generation with diffusion models. - Audio - SGLang Omni - Audio model serving for text-to-speech (TTS) and automatic speech recognition (ASR). - Speculative Decoding - SpecForge - Train draft models for speculative decoding and deploy them with SGLang. - RL and Post-Training - Miles, slime, AReaL, Tunix, verl - Training frameworks that integrate SGLang for rollout generation. - Deployment and Orchestration - SMG, RBG, llm-d, Ray Serve, NVIDIA Dynamo - Deploy and scale SGLang inference services with routing, load balancing, and cluster orchestration. - Education - Mini-SGLang, zero-to-sglang, DeepLearning.AI course - Learn inference engine design and efficient text and image generation through code and hands-on courses. Trusted by Industry and Research: >SGLang serves production workloads across AI labs, cloud platforms, enterprises, and universities. >SGLang is hosted by LMSYS, a non-profit open-source organization. Repository sgl-project/sglang Repository stats |Metric|Value| |:-|:-| |Stars|37,117| |Forks|9,547| |Last release|v0.5.21 (2026-10-02)| |Languages|Python 83%, Rust 10%, Cuda 4.3%| |Contributors|2,158| |AI contributor accounts (Claude / Codex / Cursor / Copilot)|claude (1,166 commits), cursoragent (118), Copilot (6)| |AI co-authored commits|Claude: 1,197, Cursor: 124, Copilot: 15, Codex: 10| Main contributor |Field|Value| |:-|:-| |Name|Lianmin Zheng| |GitHub|@merrymercy| |Commits to this repo|1,359 (top contributor)| |Bio|Engineer| |Company / Location|Meta / Bay Area, CA| |Website|lmzheng.net| |X / Twitter|@lm_zheng| |Followers|4,199| |Public repos|54| |On GitHub since|2015-10-13| -------------------------------------------------------------------------------------- Discover the best GitHub repos without the endless scrolling. Get top repos organized by category (AI, DevOps, Databases & more), plus key takeaways from developers' most useful comments - all in one free, twice-weekly newsletter. 👉 Get the best repos & community insights - Free