Popular GitHub Repositories for Multimodal Transformer

? Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

??? 60+ Implementations/tutorials of deep learning papers with side-by-side notes ?; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), gans(cyclegan, stylegan2, ...), ? reinforcement learning (ppo, dqn), capsnet, distillation, ... ?

Implement a ChatGPT-like LLM in PyTorch from scratch, step by step

A high-throughput and memory-efficient inference and serving engine for LLMs

The all-in-one Desktop & Docker AI application with built-in RAG, AI agents, No-code agent builder, and more.

Port of OpenAI's Whisper model in C/C++

The largest collection of PyTorch image encoders / backbones. Including train, eval, inference, export scripts, and pretrained weights -- ResNet, ResNeXT, EfficientNet, NFNet, Vision Transformer (ViT), MobileNetV4, MobileNet-V3 & V2, RegNet, DPN, CSPNet, Swin Transformer, MaxViT, CoAtNet, ConvNeXt, and more

OpenMMLab Detection Toolbox and Benchmark

Agno is a lightweight library for building Multimodal Agents. It exposes LLMs as a unified API and gives them superpowers like memory, knowledge, tools and reasoning.

Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

SOTA Open Source TTS

?? Build multimodal AI applications with cloud-native stack

? A ranked list of awesome machine learning Python libraries. Updated weekly.

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

CVPR 2025 论文和开源项目合集

MiniCPM-o 2.6: A GPT-4o Level MLLM for Vision, Speech and Multimodal Live Streaming on Your Phone

[NeurIPS 2022] Towards Robust Blind Face Restoration with Codebook Lookup Transformer

SGLang is a fast serving framework for large language models and vision language models.

:sparkles::sparkles:Latest Advances on Multimodal Large Language Models