Popular GitHub Repositories for Sglang

Production ready LLM model compression/quantization toolkit with hw accelerated inference support for both cpu/gpu via HF, vLLM, and SGLang.

我们运用SparkTTS、OrpheusTTS等先进模型，提供优质的中文语音合成和声音克隆服务。

基于SparkTTS模型，提供高质量中文语音合成与声音克隆服务。

?? Easy, advanced inference platform for large language models on Kubernetes. ? Star to support our work!

Arks是一个运行在Kubernetes上的云原生推理框架。

AI-based search engine done right

A guide to structured generation using constrained decoding

Experiments with LLMs in clouds (powered by SGLang)

DeepSeek-V3, R1 671B on 8xH100 Throughput Benchmarks

Reading notes on the open source code of AI infrastructure --- AI Infra 的开源代码阅读笔记 (LLM, DL, HPC, CUDA, SIMD...)

SgLang vs vLLM Comparison

Examples of serving LLM on Modal.

llmd is a LLMs daemonset, it provide model manager and get up and running large language models, it can use llama.cpp or vllm or sglang to running large language models.