Popular GitHub Repositories for Zero

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Two implementations of ZeRO-1 optimizer sharding in JAX

Midjourney X Instant Collage -- Collage Template + Grid + Quality Style

Fast and easy distributed model training examples.

使用自己的tokenizer继续预训练大语言模型。

Nanotron 的实验性分支版本，这是一个极简的超大型语言模型，采用 4D 并行训练法。