TextSummarization-NLP

MNBVC(Massive Never-ending BT Vast Chinese corpus)超大规模中文语料集。对标chatGPT训练的40T数据。MNBVC数据集不但包括主流文化，也包括各个小众文化甚至火星文的数据。MNBVC数据集包括新闻、作文、小说、书籍、杂志、论文、台词、帖子、wiki、古诗、歌词、商品介绍、笑话、糗事、聊天记录等一切形式的纯文本中文数据。

Module for automatic summarization of text documents and HTML pages.

AdalFlow: The library to build & auto-optimize LLM applications.

A modular RL library to fine-tune language models to human preferences

Python implementation of TextRank algorithms ("textgraphs") for phrase extraction

Efficient Retrieval Augmentation and Generation Framework

Transformers 库快速入门教程

Returns latest research results by crawling arxiv papers and summarizing abstracts. Helps you stay afloat with so many new papers everyday.

The most accurate natural language detection library for Go, suitable for short text and mixed-language text

A python package to run contextualized topic modeling. CTMs combine contextualized embeddings (e.g., BERT) with topic models to get coherent topics. Published at EACL and ACL 2021 (Bianchi et al.).