Popular GitHub Repositories for Triton Kernels

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

Efficient Triton Kernels for LLM Training

A collection of important graph embedding, classification and representation learning papers with implementations.

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

Generative AI reference workflows optimized for accelerated infrastructure and microservice architecture.

? Efficient implementations of state-of-the-art linear attention models in Torch and Triton

Quantized Attention that achieves speedups of 2.1-3.1x and 2.7-5.1x compared to FlashAttention2 and xformers, respectively, without lossing end-to-end metrics across various models.

Deep learning in Rust, with shape checked tensors and neural networks

Puzzles for learning Triton

Kernl lets you run PyTorch transformer models several times faster on GPU with a single line of code, and is designed to be easily hackable.

Add bisenetv2. My implementation of BiSeNet

Best inference performance optimization framework for HuggingFace Diffusers on NVIDIA GPUs.

LLM notes, including model inference, transformer model structure, and llm framework code analysis notes.

???A collection of some wesome public projects about Large Language Model(LLM), Vision Language Model(VLM), Vision Language Action(VLA), AI Generated Content(AIGC), the related Datasets and Applications.

More readable and flexible yolov5 with more backbone(gcn, resnet, shufflenet, moblienet, efficientnet, hrnet, swin-transformer, etc) and (cbam，dcn and so on), and tensorrt

A subset of PyTorch's neural network modules, written in Python using OpenAI's Triton.

NLP Zero to Hero in just 10 Kernels

This project is dedicated to the implementation and research of Kolmogorov-Arnold convolutional networks. The repository includes implementations of 1D, 2D, and 3D convolutions with different kernels, ResNet-like and DenseNet-like models, training code based on accelerate/PyTorch, as well as scripts for experiments with CIFAR-10 and Tiny ImageNet.

KernelBench: Can LLMs Write GPU Kernels? - Benchmark with Torch -> CUDA problems

Triton Model Analyzer is a CLI tool to help with better understanding of the compute and memory requirements of the Triton Inference Server models.