Popular GitHub Repositories for Longformer

list of efficient attention modules

Master thesis with code investigating methods for incorporating long-context reasoning in low-resource languages, without the need to pre-train from scratch. We investigated if multilingual models could inherit these properties by making it an Efficient Transformer (s.a. the Longformer architecture).

2020 AI研习社 金融用户评论分类

Convert pretrained RoBerta models to various long-document transformer models

Longformer Encoder Decoder model for the legal domain, trained for long document abstractive summarization task.

using transformers to do text classification.

LongPegasus package is used for inducing longformer self attention over base pegasus abstractive summarization model to increase the token limit and performance.

Fine-tuned Longformer for Summarization of Machine Learning Articles

A summarization website that can generate summaries from either YouTube videos or PDF files.

Project as part of COMP34812: Natural Language Understanding

Factuality check of the SemRep Predications