Popular GitHub Repositories for Amd64

A high-throughput and memory-efficient inference and serving engine for LLMs

中文LLaMA-2 & Alpaca-2大模型二期项目 + 64K超长上下文模型 (Chinese LLaMA-2 & Alpaca-2 LLMs with 64K long context models)

Speech-to-text, text-to-speech, speaker diarization, speech enhancement, and VAD using next-gen Kaldi with onnxruntime without Internet connection. Support embedded systems, Android, iOS, HarmonyOS, Raspberry Pi, RISC-V, x86_64 servers, websocket server/client, support 11 programming languages

Tengine is a lite, high performance, modular inference engine for embedded device

oneAPI Deep Neural Network Library (oneDNN)

Supercharge Your LLM with the Fastest KV Cache Layer

The Compute Library is a set of computer vision and machine learning functions optimised for both Arm CPUs and GPUs using SIMD technologies.

QualityScaler - image/video AI upscaler app

NCNN implementation of Real-ESRGAN. Real-ESRGAN aims at developing Practical Algorithms for General Image Restoration.

SHARK Studio -- Web UI for SHARK+IREE High Performance Machine Learning Distribution

Cross-platform, customizable multimedia/video processing framework. With strong GPU acceleration, heterogeneous design, multi-language support, easy to use, multi-framework compatible and high performance, the framework is ideal for transcoding, AI inference, algorithm integration, live video streaming, and more.

TinyChatEngine: On-Device LLM Inference Library

Video Frame Interpolation & Super Resolution using NVIDIA's TensorRT & Tencent's NCNN inference, beautifully crafted and packaged into a single app

中文Mixtral混合专家大模型（Chinese Mixtral MoE LLMs）

vendor independent TinyML deep learning library, compiler and inference framework microcomputers and micro-controllers

Prebuilt binary with Tensorflow Lite enabled. For RaspberryPi / Jetson Nano. Support for custom operations in MediaPipe. XNNPACK, XNNPACK Multi-Threads, FlexDelegate.

Ultra and Unified CCL

RealScaler - image/video AI upscaler app (Real-ESRGAN)

A Go (golang) library wrapping microsoft/onnxruntime.

Local LLM Server with NPU Acceleration