OpenCorpus

大规模中文自然语言处理语料 Large Scale Chinese Corpus for NLP

A collection of small corpuses of interesting data for the creation of bots and similar stuff.

Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML

A general purpose HTTP reverse proxy and forwarding tool. Now written in Go!

搜索所有中文NLP数据集，附常用英文NLP数据集

MNBVC(Massive Never-ending BT Vast Chinese corpus)超大规模中文语料集。对标chatGPT训练的40T数据。MNBVC数据集不但包括主流文化，也包括各个小众文化甚至火星文的数据。MNBVC数据集包括新闻、作文、小说、书籍、杂志、论文、台词、帖子、wiki、古诗、歌词、商品介绍、笑话、糗事、聊天记录等一切形式的纯文本中文数据。

A LITE BERT FOR SELF-SUPERVISED LEARNING OF LANGUAGE REPRESENTATIONS, 海量中文预训练ALBERT模型

Deep Learning and deep reinforcement learning research papers and some codes

?? The list of useful resources for self-study

Final Weibo Crawler Scrap Anything From Weibo, comments, weibo contents, followers, anything. The Terminator