Ant Group Unveils Two Innovative MoE Large Language Models with Significantly Reduced Training Costs

Ant Group's Ling team recently published a preprint on arXiv titled "Every FLOP Matters: Scaling a 300-billion parameter Mixture-of-Experts LING model without high-end GPUs," detailing two novel large language models: Ling-Lite and Ling-Plus. These models incorporate several innovations enabling efficient training on low-performance hardware, significantly reducing training costs.