AMD-OLMo is a series of 1 billion parameter language models trained from scratch by AMD on the AMD Instinct™ MI250 GPU, including pre-trained models, supervised fine-tuned models, and alignment models based on direct preference optimization (DPO). This series of models is based on the OLMo architecture, pre-trained on the Dolma dataset, and fine-tuned on multiple instruction datasets.
Natural Language Processing
Transformers