Popular GitHub Repositories for Listen Moe

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

SGLang is a fast serving framework for large language models and vision language models.

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and support state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in performant way.

Mixture-of-Experts for Large Vision-Language Models

? Gracefully face hCaptcha challenge with MoE(ONNX) embedded solution.

MoBA: Mixture of Block Attention for Long-Context LLMs

?? LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training (EMNLP 2024)

Tutel MoE: An Optimized Mixture-of-Experts Implementation

Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models

An open-source solution for full parameter fine-tuning of DeepSeek-V3/R1 671B, including complete code and scripts from training to inference, as well as some practical experiences and conclusions. (DeepSeek-V3/R1 满血版 671B 全参数微调的开源解决方案，包含从训练到推理的完整代码和脚本，以及实践中积累一些经验和结论。)

A toolkit for inference and evaluation of 'mixtral-8x7b-32kseqlen' from Mistral AI

中文Mixtral混合专家大模型（Chinese Mixtral MoE LLMs）

Easily create Piper text-to-speech models in any voice. Make a text-to-speech model with your own voice recordings, or use thousands of RVC voices. Works offline on a Raspberry pi. Rapidly record custom datasets for any metadata.csv file and listen to your model as it is training.

MindSpore online courses: Step into LLM

Code, Dataset, and Pretrained Models for Audio and Speech Large Language Model "Listen, Think, and Understand".

This project is a digital human that can talk and listen to you. It uses OpenAI's GPT to generate responses, OpenAI's Whisper to transcript the audio, Eleven Labs to generate voice and Rhubarb Lip Sync to generate the lip sync.

Implementation of ST-Moe, the latest incarnation of MoE after years of research at Brain, in Pytorch

Speed Always Wins: A Survey on Efficient Architectures for Large Language Models

Implementation of Soft MoE, proposed by Brain's Vision team, in Pytorch

MoH: Multi-Head Attention as Mixture-of-Head Attention