Popular GitHub Repositories for Consensus Mechanism

Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch

RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.

Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch

A concise but complete full-attention transformer with a set of promising experimental features from various papers

The Enterprise-Grade Production-Ready Multi-Agent Orchestration Framework. Website: https://swarms.ai

An ultimately comprehensive paper list of Vision Transformer/Attention, including papers, codes, and related websites

A collection of important graph embedding, classification and representation learning papers with implementations.

DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanism (SVS & TTS); AAAI 2022; Official code

Graph Attention Networks (https://arxiv.org/abs/1710.10903)

Automatic Speech Recognition (ASR), Speaker Verification, Speech Synthesis, Text-to-Speech (TTS), Language Modelling, Singing Voice Synthesis (SVS), Voice Conversion (VC)

Pytorch implementation of the Graph Attention Network model by Veličković et. al (2017, https://arxiv.org/abs/1710.10903)

It is open source ebook about TensorFlow kernel and implementation mechanism.

Show, Attend, and Tell | a PyTorch Tutorial to Image Captioning

Keras Attention Layer (Luong and Bahdanau scores).

My implementation of the original GAT paper (Veličković et al.). I've additionally included the playground.py file for visualizing the Cora dataset, GAT embeddings, an attention mechanism, and entropy histograms. I've supported both Cora (transductive) and PPI (inductive) examples!

Multilingual Automatic Speech Recognition with word-level timestamps and confidence

Reformer, the efficient Transformer, in Pytorch

FedML - The Research and Production Integrated Federated Learning Library: https://fedml.ai

To eventually become an unofficial Pytorch implementation / replication of Alphafold2, as details of the architecture get released

Bi-directional Attention Flow (BiDAF) network is a multi-stage hierarchical process that represents context at different levels of granularity and uses a bi-directional attention flow mechanism to achieve a query-aware context representation without early summarization.