Popular GitHub Repositories for Electra

Rust native ready-to-use NLP pipelines and transformer-based models (BERT, DistilBERT, GPT2,...)

ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators

Pre-trained Chinese ELECTRA（中文ELECTRA预训练模型）

Pre-trained Transformers for Arabic Language Understanding and Generation (Arabic BERT, Arabic GPT2, Arabic ELECTRA)

Pretrained ELECTRA Model for Korean

Turkish BERT/DistilBERT, ELECTRA, ConvBERT and T5 models

NLP 领域常见任务的实现，包括新词发现、以及基于pytorch的词向量、中文文本分类、实体识别、摘要文本生成、句子相似度判断、三元组抽取、预训练模型等。

Pretrain and finetune ELECTRA with fastai and huggingface. (Results of the paper replicated !)

Macadam是一个以Tensorflow(Keras)和bert4keras为基础，专注于文本分类、序列标注和关系抽取的自然语言处理工具包。支持RANDOM、WORD2VEC、FASTTEXT、BERT、ALBERT、ROBERTA、NEZHA、XLNET、ELECTRA、GPT-2等EMBEDDING嵌入; 支持FineTune、FastText、TextCNN、CharCNN、BiRNN、RCNN、DCNN、CRNN、DeepMoji、SelfAttention、HAN、Capsule等文本分类算法; 支持CRF、Bi-LSTM-CRF、CNN-LSTM、DGCNN、Bi-LSTM-LAN、Lattice-LSTM-Batch、MRC等序列标注算法。

? Korean Comments ELECTRA: ??? ??? ??? ELECTRA ??

A simple and working implementation of Electra, the fastest way to pretrain language models from scratch, in Pytorch

Build and train state-of-the-art natural language processing models using BERT

AI and Memory Wall

Pytorch-Named-Entity-Recognition-with-transformers

DBMDZ BERT, DistilBERT, ELECTRA, GPT-2 and ConvBERT models

code and data to faciliate BERT/ELECTRA for document ranking. Details refer to the paper - PARADE: Passage Representation Aggregation for Document Reranking.

We start a company-name recognition task with a small scale and low quality training data, then using skills to enhanced model training speed and predicting performance with least artificial participation. The methods we use involve lite pre-training models such as Albert-small or Electra-small with financial corpus, knowledge of distillation and multi-stage learning. The result is that we improve the recall rate of company names recognition task from 0.73 to 0.92 and get 4 times as fast as BERT-Bilstm-CRF model.

Baseline code for Korean open domain question answering(ODQA)

Electra pre-trained model using Vietnamese corpus

基于bert4keras的GLUE基准代码