Popular GitHub Repositories for Multilingual Corpus

Convert ebooks to audiobooks with chapters and metadata using dynamic AI models and voice cloning. Supports 1,107+ languages!

大规模中文自然语言处理语料 Large Scale Chinese Corpus for NLP

CodeGeeX: An Open Multilingual Code Generation Model (KDD 2023)

vits2 backbone with multilingual-bert

CodeGeeX2: A More Powerful Multilingual Code Generation Model

GLM-4 series: Open Multilingual Multimodal Chat LMs | 开源多语言多模态对话模型

High-quality multi-lingual text-to-speech library by MyShell.ai. Support English, Spanish, French, Chinese, Japanese and Korean.

Multilingual Voice Understanding Model

An Open Source Python alternative to NotebookLM's podcast feature: Transforming Multimodal Content into Captivating Multilingual Audio Conversations with GenAI

A collection of small corpuses of interesting data for the creation of bots and similar stuff.

The Abstraction and Reasoning Corpus

Gradio WebUI for creators and developers, featuring key TTS (Edge-TTS, kokoro) and zero-shot Voice Cloning (E2E, F5-TTS, CosyVoice), with Whisper audio processing, RVC voice changer, YouTube download, UVR5 vocal isolation, and multilingual translation.

搜索所有中文NLP数据集，附常用英文NLP数据集

??AutoDev: The AI-powered coding wizard（AI 驱动编程助手）with multilingual support ?, auto code generation ??, and a helpful bug-slaying assistant ?! Customizable prompts ? and a magic Auto Dev/Testing/Document/Agent feature ? included! ?

A LITE BERT FOR SELF-SUPERVISED LEARNING OF LANGUAGE REPRESENTATIONS, 海量中文预训练ALBERT模型

MNBVC(Massive Never-ending BT Vast Chinese corpus)超大规模中文语料集。对标chatGPT训练的40T数据。MNBVC数据集不但包括主流文化，也包括各个小众文化甚至火星文的数据。MNBVC数据集包括新闻、作文、小说、书籍、杂志、论文、台词、帖子、wiki、古诗、歌词、商品介绍、笑话、糗事、聊天记录等一切形式的纯文本中文数据。

LightSeq: A High Performance Library for Sequence Processing and Generation

Deep Learning and deep reinforcement learning research papers and some codes

Kandinsky 2 — multilingual text2image latent diffusion model

Go efficient multilingual NLP and text segmentation; support English, Chinese, Japanese and others.