Product Finder

Product Submit

AI Models Finder

MCP Servers

MCP Client

MCP Inspector

Case Tutorials

Latest AI News

AI Daily Brief

local-llm-workbench

? A comprehensive toolkit for benchmarking, optimizing, and deploying local Large Language Models. Includes performance testing tools, optimized configurations for CPU/GPU/hybrid setups, and detailed guides to maximize LLM performance on your hardware.

context-window-scaling cpu-inference cuda gpu-acceleration hybrid-inference inference-optimization llama-cpp llm-benchmarking llm-deployment local-llm

Creat：2025-03-27T09:13:00

Update：2025-03-27T09:47:23

2

Stars

0

Stars Increase

Related projects

Gpt4all

GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.

Zustand

? Bear necessities for state management in React

Vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

Whisper.Cpp

Port of OpenAI's Whisper model in C/C++

ColossalAI

Making large AI models cheaper, faster and more accessible

DeepSpeed

billion-parameters

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Ray

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

ChatTTS

A generative speech model for daily dialogue.

Mediapipe

Cross-platform, customizable ML solutions for live and streaming media.

So Vits Svc

SoftVC VITS Singing Voice Conversion