Popular GitHub Repositories for Localllama

? Kubectl plugin to create manifests with LLMs

在 llama.cpp（或任何本地兼容 OpenAPI 的服务器）上进行模型交换

The easiest way to use the Ollama API in .NET

Like grep but for natural language questions. Based on Mistral 7B or Mixtral 8x7B.

[NeurIPS 2024] KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization

AubAI brings you on-device gen-AI capabilities, including offline text generation and more, directly within your app.

Social and customizable AI writing assistant! ??

A local and uncensored AI entity.

LLM RAG Application with Cross-Encoders Re-ranking for YouTube video ?

Use your open source local model from the terminal

? LocalLLaMA Archive — Community-powered static archive for r/LocalLLaMA

Copilot hack for running local copilot without auth and proxying

Speed-oriented LocalLLaMA App on Android

A chat interface in Streamlit for LLMs using Ollama.

Lightweight Python tool using Optuna for tuning llama.cpp flags: towards optimal tok/s for your machine