HomeAI Tutorial

minLLMTrain

Public

Minimal yet high performant code for pretraining llms. Attempts to implement some SOTA features. Implements training through: Deepspeed, Megatron-LM, and FSDP. WIP

Creat2024-02-01T21:03:27
Update2025-01-06T06:16:58
6
Stars
0
Stars Increase