Popular GitHub Repositories for Megatron Lm

Annotations of the interesting ML papers I read

A LLaMA1/LLaMA12 Megatron implement.

Best practices for testing advanced Mixtral, DeepSeek, and Qwen series MoE models using Megatron Core MoE.

Minimal yet high performant code for pretraining llms. Attempts to implement some SOTA features. Implements training through: Deepspeed, Megatron-LM, and FSDP. WIP

Wrapped Megatron: As User-Friendly as HuggingFace, As Powerful as Megatron-LM | Megatron封装：和HuggingFace一样方便，和Megatron-LM一样强大

Running Large Language Model easily.