Popular GitHub Repositories for Llama Inference Server

Dify is an open-source LLM app development platform. Dify's intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features and more, letting you quickly go from prototype to production.

? Light and Fast AI Assistant. Support: Web | iOS | MacOS | Android | Linux | Windows

GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.

For developers, who are building real-time data-driven applications, Redis is the preferred, fastest, and most feature-rich cache, data structure server, and document and vector query engine.

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

A high-throughput and memory-efficient inference and serving engine for LLMs

The all-in-one Desktop & Docker AI application with built-in RAG, AI agents, No-code agent builder, and more.

Finetune Llama 3.3, DeepSeek-R1 & Reasoning LLMs 2x faster with 70% less memory! ?

下一代 Node.js 和 TypeScript 对象关系映射 (ORM) 工具，支持 PostgreSQL、MySQL、MariaDB、SQL Server、SQLite、MongoDB 和 CockroachDB 数据库。

Port of OpenAI's Whisper model in C/C++

Making large AI models cheaper, faster and more accessible

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

Opiniated RAG for integrating GenAI in your apps ? Focus on your product rather than the RAG. Easy integration in existing products with customisation! Any LLM: GPT4, Groq, Llama. Any Vectorstore: PGVector, Faiss. Any Files. Anyway you want.

A generative speech model for daily dialogue.

aider is AI pair programming in your terminal

Langchain-Chatchat（原Langchain-ChatGLM）基于 Langchain 与 ChatGLM, Qwen 与 Llama 等语言模型的 RAG 与 Agent 应用 | Langchain-Chatchat (formerly langchain-ChatGLM), local knowledge based LLM (like ChatGLM, Qwen and Llama) RAG and Agent app with langchain

Jan is an open source alternative to ChatGPT that runs 100% offline on your computer

The largest collection of PyTorch image encoders / backbones. Including train, eval, inference, export scripts, and pretrained weights -- ResNet, ResNeXT, EfficientNet, NFNet, Vision Transformer (ViT), MobileNetV4, MobileNet-V3 & V2, RegNet, DPN, CSPNet, Swin Transformer, MaxViT, CoAtNet, ConvNeXt, and more

:robot: The free, Open Source alternative to OpenAI, Claude and others. Self-hosted and local-first. Drop-in replacement for OpenAI, running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. Features: Generate Text, Audio, Video, Images, Voice Cloning, Distributed, P2P inference