Linear-Attention-Recurrent-Neural-Network

Public

A recurrent attention module consisting of an LSTM cell which can query its own past cell states by the means of windowed multi-head attention. The formulas are derived from the BN-LSTM and the Transformer Network. The LARNN cell with attention can be easily used inside a loop on the cell state, just like any other RNN. (LARNN)

attention-is-all-you-need attention-mechanism attention-model lstm pytorch recurrent-neural-networks rnn

Creat：2018-05-03T11:46:15

Update：2024-11-13T21:24:12

https://www.neuraxio.com/

145

Stars

Stars Increase

Related projects

Transformers

Hot

bert

? Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

147733

2年前

+59today

Generative Ai For Beginners

Hot

21 Lessons, Get Started Building with Generative AI ? https://microsoft.github.io/generative-ai-for-beginners/

93782

5个月前

+85today

Annotated_deep_learning_paper_implementations

attention

??? 60+ Implementations/tutorials of deep learning papers with side-by-side notes ?; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), gans(cyclegan, stylegan2, ...), ? reinforcement learning (ppo, dqn), capsnet, distillation, ... ?

62216

4个月前

+14today