BeamSD
PublicThis is a Python package for accelerating the inference of Large Language Models (LLMs) by Speculative Decoding (SD), especially for Beam Search.
Creat:2024-10-16T21:06:02
Update:2024-11-27T21:28:26
2
Stars
0
Stars Increase
This is a Python package for accelerating the inference of Large Language Models (LLMs) by Speculative Decoding (SD), especially for Beam Search.