Popular GitHub Repositories for Bert Pre Training

? Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

Making large AI models cheaper, faster and more accessible

Learn how to design, develop, deploy and iterate on production-grade ML applications.

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

The largest collection of PyTorch image encoders / backbones. Including train, eval, inference, export scripts, and pretrained weights -- ResNet, ResNeXT, EfficientNet, NFNet, Vision Transformer (ViT), MobileNetV4, MobileNet-V3 & V2, RegNet, DPN, CSPNet, Swin Transformer, MaxViT, CoAtNet, ConvNeXt, and more

PArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice （『飞桨』核心框架，深度学习&机器学习高性能单机、分布式训练和跨平台部署）

This repository is primarily maintained by Omar Santos (@santosomar) and includes thousands of resources related to ethical hacking, bug bounties, digital forensics and incident response (DFIR), artificial intelligence security, vulnerability research, exploit development, reverse engineering, and more.

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

AI orchestration framework to build customizable, production-ready LLM applications. Connect components (models, vector DBs, file converters) to pipelines or agents that can interact with your data. With advanced retrieval methods, it's best suited for building RAG, question answering, semantic search or conversational agent chatbots.

本项目旨在分享大模型相关技术原理以及实战经验（大模型工程化、大模型应用落地）

A WebGL accelerated JavaScript library for training and deploying ML models.

中文LLaMA&Alpaca大语言模型+本地CPU/GPU训练部署 (Chinese LLaMA & Alpaca LLMs)

The Unity Machine Learning Agents Toolkit (ML-Agents) is an open-source project that enables games and simulations to serve as environments for training intelligent agents using deep reinforcement learning and imitation learning.

ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator

Multi-lingual large voice generation model, providing inference, training and deployment full-stack ability.

Natural Language Processing Tutorial for Deep Learning Researchers

Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.

Machine Learning Engineering Open Book

RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.

PaddleFormers is an easy-to-use library of pre-trained large language model zoo based on PaddlePaddle.