Popular GitHub Repositories for Model Inference

? Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

Dify is an open-source LLM app development platform. Dify's intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features and more, letting you quickly go from prototype to production.

21 Lessons, Get Started Building with Generative AI ? https://microsoft.github.io/generative-ai-for-beginners/

GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.

Implement a ChatGPT-like LLM in PyTorch from scratch, step by step

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

A high-throughput and memory-efficient inference and serving engine for LLMs

1 min voice data can also be used to train a good TTS model! (few shot voice cloning)

Port of OpenAI's Whisper model in C/C++

?? - a deep learning toolkit for Text-to-Speech, battle-tested in research and production

Making large AI models cheaper, faster and more accessible

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

OpenAssistant is a chat-based assistant that understands tasks, can interact with third-party systems, and retrieve information dynamically to do so.

A generative speech model for daily dialogue.

The largest collection of PyTorch image encoders / backbones. Including train, eval, inference, export scripts, and pretrained weights -- ResNet, ResNeXT, EfficientNet, NFNet, Vision Transformer (ViT), MobileNetV4, MobileNet-V3 & V2, RegNet, DPN, CSPNet, Swin Transformer, MaxViT, CoAtNet, ConvNeXt, and more

:robot: The free, Open Source alternative to OpenAI, Claude and others. Self-hosted and local-first. Drop-in replacement for OpenAI, running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. Features: Generate Text, Audio, Video, Images, Voice Cloning, Distributed, P2P inference

Instant voice cloning by MIT and MyShell. Audio foundation model.

Easily train a good VC model with voice data <= 10 mins!

Cross-platform, customizable ML solutions for live and streaming media.