Popular GitHub Repositories for Llm Serving

Elixir implementation of a LangChain style framework that lets Elixir projects integrate with and leverage LLMs.

Dify is an open-source LLM app development platform. Dify's intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features and more, letting you quickly go from prototype to production.

GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.

为GPT/GLM等LLM大语言模型提供实用化交互接口，特别优化论文阅读/润色/写作体验，模块化设计，支持自定义快捷按钮&函数插件，支持Python和C++等项目剖析&自译解功能，PDF/LaTex论文翻译&总结功能，支持并行问询多种LLM模型，支持chatglm3等本地模型。接入通义千问, deepseekcoder, 讯飞星火, 文心一言, llama2, rwkv, claude2, moss等。

Make websites accessible for AI agents

Implement a ChatGPT-like LLM in PyTorch from scratch, step by step

? The Multi-Agent Framework: First AI Software Company, Towards Natural Language Programming

Collection of awesome LLM apps with AI Agents and RAG using OpenAI, Anthropic, Gemini and opensource models.

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

A high-throughput and memory-efficient inference and serving engine for LLMs

A programming framework for agentic AI ? PyPi: autogen-agentchat Discord: https://aka.ms/autogen-discord Office Hour: https://aka.ms/autogen-officehour

The all-in-one Desktop & Docker AI application with built-in RAG, AI agents, No-code agent builder, and more.

? Turn entire websites into LLM-ready markdown or structured data. Scrape, crawl and extract with a single API.

LlamaIndex is the leading framework for building LLM-powered agents over your data.

Finetune Llama 3.3, DeepSeek-R1 & Reasoning LLMs 2x faster with 70% less memory! ?

Drag & drop UI to build your customized LLM flow

利用AI大模型，一键生成高清短视频 Generate short videos with one click using AI LLM.

基于大模型搭建的聊天机器人，同时支持 微信公众号、企业微信应用、飞书、钉钉 等接入，可选择GPT3.5/GPT-4o/GPT-o1/ DeepSeek/Claude/文心一言/讯飞星火/通义千问/ Gemini/GLM-4/Claude/Kimi/LinkAI，能处理文本、语音和图片，访问操作系统和互联网，支持基于自有知识库进行定制企业智能客服。

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

Opiniated RAG for integrating GenAI in your apps ? Focus on your product rather than the RAG. Easy integration in existing products with customisation! Any LLM: GPT4, Groq, Llama. Any Vectorstore: PGVector, Faiss. Any Files. Anyway you want.