Popular GitHub Repositories for Linear Attention

??? 60+ Implementations/tutorials of deep learning papers with side-by-side notes ?; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), gans(cyclegan, stylegan2, ...), ? reinforcement learning (ppo, dqn), capsnet, distillation, ... ?

100 Days of ML Coding

Machine Learning From Scratch. Bare bones NumPy implementations of machine learning models and algorithms with a focus on accessibility. Aims to cover everything from linear regression to deep learning.

Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch

Machine learning, in numpy

Natural Language Processing Tutorial for Deep Learning Researchers

RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.

MIT Deep Learning Book in PDF format (complete and parts) by Ian Goodfellow, Yoshua Bengio and Aaron Courville

A PyTorch implementation of the Transformer model in "Attention is All You Need".

WebGL-accelerated ML // linear algebra // automatic differentiation for JavaScript.

CUDA Templates for Linear Algebra Subroutines

Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture. Basically ChatGPT but with PaLM

BertViz: Visualize Attention in NLP Models (BERT, GPT2, BART, etc.)

Detailed and tailored guide for undergraduate students or anybody want to dig deep into the field of AI with solid foundation.

H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.

中文LLaMA-2 & Alpaca-2大模型二期项目 + 64K超长上下文模型 (Chinese LLaMA-2 & Alpaca-2 LLMs with 64K long context models)

Official release of InternLM series (InternLM, InternLM2, InternLM2.5, InternLM3).

Code for Tensorflow Machine Learning Cookbook

Statistical Machine Intelligence & Learning Engine

The GitHub repository for the paper "Informer" accepted by AAAI 2021.