Popular GitHub Repositories for Efficient Tokenizer

? Empowering everyone to build reliable and efficient smart contracts.

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

A high-throughput and memory-efficient inference and serving engine for LLMs

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

? The largest hub of ready-to-use datasets for ML models with fast, easy-to-use and efficient data manipulation tools

??HivisionIDPhotos: a lightweight and efficient AI ID photos tools. 一个轻量级的AI证件照制作算法。

Unsupervised text tokenizer for Neural Network-based text generation.

OneFlow is a deep learning framework designed to be user-friendly, scalable and efficient.

? 卡卡字幕助手 | VideoCaptioner - 基于 LLM 的智能字幕助手 - 视频字幕生成、断句、校正、字幕翻译全流程处理！- A powered tool for easy and efficient video subtitling.

A robust, efficient, low-latency speech-to-text library with advanced voice activity detection, wake word activation and instant transcription.

Chinese version of GPT2 training code, using BERT tokenizer.

Efficient Triton Kernels for LLM Training

Official PyTorch implementation of PTS/PSRN: Fast and efficient symbolic expression discovery through parallelized tree search. Evaluates millions of expressions simultaneously on GPU with automated subtree reuse.

An efficient, flexible and full-featured toolkit for fine-tuning LLM (InternLM2, Llama3, Phi3, Qwen, Mistral, ...)

SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformer

Efficient AI Backbones including GhostNet, TNT and MLP, developed by Huawei Noah's Ark Lab.

Massively Parallel Deep Reinforcement Learning. ?

A language for constraint-guided and efficient LLM programming.

A unified, comprehensive and efficient recommendation library

A highly efficient implementation of Gaussian Processes in PyTorch