SentiArt

MNBVC(Massive Never-ending BT Vast Chinese corpus)超大规模中文语料集。对标chatGPT训练的40T数据。MNBVC数据集不但包括主流文化，也包括各个小众文化甚至火星文的数据。MNBVC数据集包括新闻、作文、小说、书籍、杂志、论文、台词、帖子、wiki、古诗、歌词、商品介绍、笑话、糗事、聊天记录等一切形式的纯文本中文数据。

The most accurate natural language detection library for Go, suitable for short text and mixed-language text

A python package to run contextualized topic modeling. CTMs combine contextualized embeddings (e.g., BERT) with topic models to get coherent topics. Published at EACL and ACL 2021 (Bianchi et al.).

A deep dive into embeddings starting from fundamentals

Rasa UI is a frontend for the Rasa Framework

The most accurate natural language detection library for Rust, suitable for short text and mixed-language text

skweak: A software toolkit for weak supervision applied to NLP tasks

We introduced a new model designed for the Code generation task. Its test accuracy on the HumanEval base dataset surpasses that of GPT-4 Turbo (April 2024) and GPT-4o.

BookNLP, a natural language processing pipeline for books

Toolkit for fine-tuning, ablating and unit-testing open-source LLMs.