Popular GitHub Repositories for Decentralized Training

Making large AI models cheaper, faster and more accessible

Learn how to design, develop, deploy and iterate on production-grade ML applications.

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

AI-Powered Photos App for the Decentralized Web ???

The largest collection of PyTorch image encoders / backbones. Including train, eval, inference, export scripts, and pretrained weights -- ResNet, ResNeXT, EfficientNet, NFNet, Vision Transformer (ViT), MobileNetV4, MobileNet-V3 & V2, RegNet, DPN, CSPNet, Swin Transformer, MaxViT, CoAtNet, ConvNeXt, and more

PArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice （『飞桨』核心框架，深度学习&机器学习高性能单机、分布式训练和跨平台部署）

This repository is primarily maintained by Omar Santos (@santosomar) and includes thousands of resources related to ethical hacking, bug bounties, digital forensics and incident response (DFIR), artificial intelligence security, vulnerability research, exploit development, reverse engineering, and more.

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

A WebGL accelerated JavaScript library for training and deploying ML models.

本项目旨在分享大模型相关技术原理以及实战经验（大模型工程化、大模型应用落地）

The Unity Machine Learning Agents Toolkit (ML-Agents) is an open-source project that enables games and simulations to serve as environments for training intelligent agents using deep reinforcement learning and imitation learning.

ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator

Self-Hosting Guide. Learn all about locally hosting (on premises & private web servers) and managing software applications by yourself or your organization. Including Cloud, LLMs, WireGuard, Automation, Home Assistant, and Networking.

Multi-lingual large voice generation model, providing inference, training and deployment full-stack ability.

Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.

Machine Learning Engineering Open Book

RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.

Ongoing research training transformer models at scale

? Easy-to-use and powerful NLP and LLM library with ? Awesome model zoo, supporting wide-range of NLP tasks from research to industrial applications, including ?Text Classification, ? Neural Search, ? Question Answering, ?? Information Extraction, ? Document Intelligence, ? Sentiment Analysis etc.

Collection of scripts to aggregate image data for the purposes of training an NSFW Image Classifier