Popular GitHub Repositories for Modality Inversion

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

:sparkles::sparkles:Latest Advances on Multimodal Large Language Models

? Scalable embedding, reasoning, ranking for images and sentences with CLIP

a state-of-the-art-level open visual language model | 多模态预训练模型

The Enterprise-Grade Production-Ready Multi-Agent Orchestration Framework. Website: https://swarms.ai

Simple command line tool for text to image generation using OpenAI's CLIP and Siren (Implicit neural representation network). Technique was originally created by https://twitter.com/advadnoun

Align Anything: Training All-modality Model with Feedback

Official repo for "Mini-Gemini: Mining the Potential of Multi-modality Vision Language Models"

? Otter, a multi-modal model based on OpenFlamingo (open-sourced version of DeepMind's Flamingo), trained on MIMIC-IT and showcasing improved instruction-following and in-context learning ability.

Implementation of Dreambooth (https://arxiv.org/abs/2208.12242) by way of Textual Inversion (https://arxiv.org/abs/2208.01618) for Stable Diffusion (https://arxiv.org/abs/2112.10752). Tweaks focused on training faces, objects, and styles.

InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactions

Open-source evaluation toolkit of large multi-modality models (LMMs), support 220+ LMMs, 80+ benchmarks

Lumina-T2X is a unified framework for Text to Any Modality Generation

[ICLR 2024] Official implementation of " ? Time-LLM: Time Series Forecasting by Reprogramming Large Language Models"

collection of diffusion model papers categorized by their subareas

Awesome Person Re-identification

[TPAMI 2022] GAN Inversion: A Survey

Official Implementation for "HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editing" (CVPR 2022) https://arxiv.org/abs/2111.15666

GAN-based Mel-Spectrogram Inversion Network for Text-to-Speech Synthesis

Official Implementation for "Pivotal Tuning for Latent-based editing of Real Images" (ACM TOG 2022) https://arxiv.org/abs/2106.05744