Popular GitHub Repositories for Audio Captioning

:robot: The free, Open Source alternative to OpenAI, Claude and others. Self-hosted and local-first. Drop-in replacement for OpenAI, running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. Features: Generate Text, Audio, Video, Images, Voice Cloning, Distributed, P2P inference

Instant voice cloning by MIT and MyShell. Audio foundation model.

Easily train a good VC model with voice data <= 10 mins!

Cross-platform, customizable ML solutions for live and streaming media.

? Diffusers: State-of-the-art diffusion models for image, video, and audio generation in PyTorch and FLAX.

SoftVC VITS Singing Voice Conversion

Deezer source separation library including pretrained models.

GUI for a Vocal Remover that uses Deep Neural Networks.

Multi-lingual large voice generation model, providing inference, training and deployment full-stack ability.

Buzz transcribes and translates audio offline on your personal computer. Powered by OpenAI's Whisper.

[CVPR 2023] SadTalker：Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animation

Awesome pre-trained models toolkit based on PaddlePaddle. (400+ models including Image, Text, Audio, Video and Cross-Modal with Easy Inference & Serving)

A Fundamental End-to-End Speech Recognition Toolkit and Open Source SOTA Pretrained Models, Supporting Speech Recognition, Voice Activity Detection, Text Post-processing etc.

A PyTorch-based Speech Toolkit

AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Head

Amphion (/?m?fa??n/) is a toolkit for Audio, Music, and Speech Generation. Its purpose is to support reproducible research and help junior researchers and engineers get started in the field of audio, music, and speech generation research and development.

Use PEFT or Full-parameter to finetune 450+ LLMs (Qwen2.5, InternLM3, GLM4, Llama3.3, Mistral, Yi1.5, Baichuan2, DeepSeek-R1, ...) and 150+ MLLMs (Qwen2.5-VL, Qwen2-Audio, Llama3.2-Vision, Llava, InternVL2.5, MiniCPM-V-2.6, GLM4v, Xcomposer2.5, Yi-VL, DeepSeek-VL2, Phi3.5-Vision, GOT-OCR2, ...).

Speech recognition module for Python, supporting several engines and APIs, online and offline.

A sound cloning tool with a web interface, using your voice or any sound to record audio / 一个带web界面的声音克隆工具，使用你的音色或任意声音来录制音频

Code for the paper "Jukebox: A Generative Model for Music"