Popular GitHub Repositories for Multimedia

Meta-Transformer for Unified Multimodal Learning

Cross-platform, customizable multimedia/video processing framework. With strong GPU acceleration, heterogeneous design, multi-language support, easy to use, multi-framework compatible and high performance, the framework is ideal for transcoding, AI inference, algorithm integration, live video streaming, and more.

An open-source computer vision framework to build and deploy apps in minutes

"Pop Music Transformer: Beat-based Modeling and Generation of Expressive Pop Piano Compositions", ACM Multimedia 2020

This repository is a home to Intel? Deep Learning Streamer (Intel? DL Streamer) Pipeline Framework. Pipeline Framework is a streaming media analytics framework, based on GStreamer* multimedia framework, for creating complex media analytics pipelines.

Multi-modality pre-training

ACM Multimedia 2023: DocDiff: Document Enhancement via Residual Diffusion Models. Also contains 1597 red seals in Chinese scenes, along with their corresponding binary masks.

A microservice-based AI education platform for children that integrates LLMs, image generation, and speech synthesis to provide personalized storybook creation, intelligent conversational learning, and multimedia content generation.

Multiple notebooks which allow the use of various machine learning methods to generate or modify multimedia content

[ACM TIST 2025] GenAI in Fashion: Overview (??????? Also, a curated list of cool resources about GenAI-Fashion, including ?papers, ?workshops, ?companies & products, ...)

Useful Toolbox for Anomaly Detection

Coarse-to-Fine CNN for Image Super-Resolution (IEEE Transactions on Multimedia,2021)

[TCSVT] CorrI2P: Deep Image-to-Point Cloud Registration via Dense CorrespondenceThe code of CorrI2P

Python toolkit for pluggable algorithms and data structures for multimedia-based machine learning.

Hsin-Rung Chou, Jia-Hong Lee, Yi-Ming Chan, and Chu-Song Chen, "Data-specific Adaptive Threshold for Face Recognition and Authentication," IEEE International Conference on Multimedia Information Processing and Retrieval, MIPR 2019

[ACM MM'2024]"DiffMM: Multi-Modal Diffusion Model for Recommendation"

TengineGst is a streaming media analytics framework, based on GStreamer multimedia framework, for creating varied complex media analytics pipelines. It ensures pipeline interoperability and provides optimized media, and inference operations using Tengine Toolkit Inference Engine backend, across varied architecture - CPU, iGPU and VPU.

[SOICT 2024] LLM-Powered Video Search: A Comprehensive Multimedia Retrieval System

Cottontail DB is a column store vector database aimed at multimedia retrieval. It allows for classical boolean as well as vector-space retrieval (nearest neighbour search) used in similarity search using a unified data and query model.

Official implementation of SGDiff (ACM MM '23)