Popular GitHub Repositories for Speech Understanding

? Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

RAGFlow is an open-source RAG (Retrieval-Augmented Generation) engine based on deep document understanding.

Clone a voice in 5 seconds to generate arbitrary speech in real-time

1 min voice data can also be used to train a good TTS model! (few shot voice cloning)

Port of OpenAI's Whisper model in C/C++

?? - a deep learning toolkit for Text-to-Speech, battle-tested in research and production

A generative speech model for daily dialogue.

?AI拟声: 5秒内克隆您的声音并生成任意语音内容 Clone a voice in 5 seconds to generate arbitrary speech in real-time

Instant voice cloning by MIT and MyShell. Audio foundation model.

OpenPose: Real-time multi-person keypoint detection library for body, face, hands, and foot estimation

aka "Bayesian Methods for Hackers": An introduction to Bayesian methods + probabilistic programming with a computation/understanding-first, mathematics-second point of view. All in pure Python ;)

SoftVC VITS Singing Voice Conversion

DeepSpeech is an open source embedded (offline, on-device) speech-to-text engine which can run in real time on devices ranging from a Raspberry Pi 4 to high power GPU servers.

MiniCPM-V 4.5: A GPT-4o Level MLLM for Single Image, Multi Image and Video Understanding on Your Phone

? The largest hub of ready-to-use datasets for ML models with fast, easy-to-use and efficient data manipulation tools

MiniCPM-o 2.6: A GPT-4o Level MLLM for Vision, Speech and Multimodal Live Streaming on Your Phone

WhisperX: Automatic Speech Recognition with Word-level Timestamps (& Diarization)

Multi-lingual large voice generation model, providing inference, training and deployment full-stack ability.

Grounded SAM: Marrying Grounding DINO with Segment Anything & Stable Diffusion & Recognize Anything - Automatically Detect , Segment and Generate Anything

? Leon is your open-source personal assistant.