llamacpp

A high-throughput and memory-efficient inference and serving engine for LLMs

Jan is an open source alternative to ChatGPT that runs 100% offline on your computer

Your AI second brain. Self-hostable. Get answers from the web or your docs. Build custom agents, schedule automations, do deep research. Turn any online or local LLM into your personal, autonomous AI (gpt, claude, gemini, llama, qwen, mistral). Get started - free.

SGLang is a fast serving framework for large language models and vision language models.

kaldi-asr/kaldi is the official location of the Kaldi project.

Unified framework for building enterprise RAG pipelines with small, specialized models

Open3D: A Modern Library for 3D Data Processing

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and support state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in performant way.

A self-hosted, offline, ChatGPT-like chatbot. Powered by Llama 2. 100% private, with no data leaving your device. New: Code Llama support!

NumPy aware dynamic Python compiler using LLVM