Popular GitHub Repositories for Llm Infernece

Elixir implementation of a LangChain style framework that lets Elixir projects integrate with and leverage LLMs.

Dify is an open-source LLM app development platform. Dify's intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features and more, letting you quickly go from prototype to production.

Collection of awesome LLM apps with AI Agents and RAG using OpenAI, Anthropic, Gemini and opensource models.

Implement a ChatGPT-like LLM in PyTorch from scratch, step by step

GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.

Make websites accessible for AI agents

为GPT/GLM等LLM大语言模型提供实用化交互接口，特别优化论文阅读/润色/写作体验，模块化设计，支持自定义快捷按钮&函数插件，支持Python和C++等项目剖析&自译解功能，PDF/LaTex论文翻译&总结功能，支持并行问询多种LLM模型，支持chatglm3等本地模型。接入通义千问, deepseekcoder, 讯飞星火, 文心一言, llama2, rwkv, claude2, moss等。

? Turn entire websites into LLM-ready markdown or structured data. Scrape, crawl and extract with a single API.

A high-throughput and memory-efficient inference and serving engine for LLMs

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

? The Multi-Agent Framework: First AI Software Company, Towards Natural Language Programming

A programming framework for agentic AI ? PyPi: autogen-agentchat Discord: https://aka.ms/autogen-discord Office Hour: https://aka.ms/autogen-officehour

The all-in-one Desktop & Docker AI application with built-in RAG, AI agents, No-code agent builder, and more.

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

Finetune Llama 3.3, DeepSeek-R1 & Reasoning LLMs 2x faster with 70% less memory! ?

利用AI大模型，一键生成高清短视频 Generate short videos with one click using AI LLM.

Ready-to-run cloud templates for RAG, AI pipelines, and enterprise search with live data. ?Docker-friendly.?Always in sync with Sharepoint, Google Drive, S3, Kafka, PostgreSQL, real-time data APIs, and more.

Drag & drop UI to build your customized LLM flow

LlamaIndex is the leading framework for building LLM-powered agents over your data.

The Memory layer for AI Agents