batch-inference
PublicDynamic batching library for Deep Learning inference. Tutorials for LLM, GPT scenarios.
Creat:2023-04-17T15:15:38
Update:2025-02-27T04:11:37
https://microsoft.github.io/batch-inference/
101
Stars
0
Stars Increase
Dynamic batching library for Deep Learning inference. Tutorials for LLM, GPT scenarios.