Popular GitHub Repositories for Megabyte

Implementation of MEGABYTE, Predicting Million-byte Sequences with Multiscale Transformers, in Pytorch

Implementation of MambaByte in "MambaByte: Token-free Selective State Space Model" in Pytorch and Zeta

A PyTorch implementation of MEGABYTE. This multi-scale transformer architecture has the excellent features of tokenization-free and sub-quadratic attention. The paper link: https://arxiv.org/abs/2305.07185