livebatch
PublicA lightweight, framework-agnostic middleware that dynamically batches inference requests in real time to maximize GPU/TPU utilization.
Creat:2025-05-23T15:24:34
Update:2025-07-25T19:42:14
0
Stars
0
Stars Increase
A lightweight, framework-agnostic middleware that dynamically batches inference requests in real time to maximize GPU/TPU utilization.