AIbase
Product LibraryTool Navigation

LLM-Inference-Serving

Public

This repository demonstrates LLM execution on CPUs using packages like llamafile, emphasizing low-latency, high-throughput, and cost-effective benefits for inference and serving.

Creat2023-12-02T20:31:39
Update2024-10-03T22:10:24
9
Stars
0
Stars Increase

Related projects