Popular GitHub Repositories for Long Context Attention

??? 60+ Implementations/tutorials of deep learning papers with side-by-side notes ?; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), gans(cyclegan, stylegan2, ...), ? reinforcement learning (ppo, dqn), capsnet, distillation, ... ?

The Memory layer for AI Agents

Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch

LLM based autonomous agent that conducts deep local and web research on any topic and generates a long report with citations.

Letta (formerly MemGPT) is the stateful agents framework with memory, reasoning, and context management.

Machine learning, in numpy

:sparkles::sparkles:Latest Advances on Multimodal Large Language Models

Natural Language Processing Tutorial for Deep Learning Researchers

RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.

The official GitHub page for the survey paper "A Survey of Large Language Models".

What are the principles we can use to build LLM-powered software that is actually good enough to put in the hands of production customers?

A PyTorch implementation of the Transformer model in "Attention is All You Need".

Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture. Basically ChatGPT but with PaLM

The most reliable AI agent framework that supports MCP.

BertViz: Visualize Attention in NLP Models (BERT, GPT2, BART, etc.)

Developer-friendly, serverless vector database for AI applications. Easily add long-term memory to your LLM apps!

中文LLaMA-2 & Alpaca-2大模型二期项目 + 64K超长上下文模型 (Chinese LLaMA-2 & Alpaca-2 LLMs with 64K long context models)

Official release of InternLM series (InternLM, InternLM2, InternLM2.5, InternLM3).

Build effective agents using Model Context Protocol and simple workflow patterns

The GitHub repository for the paper "Informer" accepted by AAAI 2021.