Efficient LLM
An efficient solution for LLM inference on Intel GPUs.
CommonProductProductivityLLMInference
This is an efficient LLM inference solution implemented on Intel GPUs. By simplifying the LLM decoder layer, utilizing segment KV caching strategies, and implementing a custom Scaled-Dot-Product-Attention kernel, this solution achieves up to 7x lower token latency and 27x higher throughput on Intel GPUs compared to the standard HuggingFace implementation. For detailed features, advantages, pricing, and positioning information, please refer to the official website.
Efficient LLM Visit Over Time
Monthly Visits
27175375
Bounce Rate
44.30%
Page per Visit
5.8
Visit Duration
00:04:57
