Popular GitHub Repositories for Mllm

《开源大模型食用指南》针对中国宝宝量身打造的基于Linux环境快速微调（全参数/Lora）、部署国内外开源大模型（LLM）/多模态大模型（MLLM）教程

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

MiniCPM-o 2.6: A GPT-4o Level MLLM for Vision, Speech and Multimodal Live Streaming on Your Phone

Reasoning in LLMs: Papers and Resources, including Chain-of-Thought, OpenAI o1, and DeepSeek-R1 ?

InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactions

? Sa2VA: Marrying SAM2 with LLaVA for Dense Grounded Understanding of Images and Videos

A novel Multimodal Large Language Model (MLLM) architecture, designed to structurally align visual and textual embeddings.

OpenEMMA, a permissively licensed open source "reproduction" of Waymo’s EMMA model.

Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey

??Woodpecker: Hallucination Correction for Multimodal Large Language Models

JarvisArt: Liberating Human Artistic Creativity via an Intelligent Photo Retouching Agent

[ECCV2024] Grounded Multimodal Large Language Model with Localized Visual Tokenization

Custom ComfyUI nodes for Vision Language Models, Large Language Models, Image to Music, Text to Music, Consistent and Random Creative Prompt Generation

??? A curated list of papers on LLMs-based multimodal generation (image, video, 3D and audio).

Personal Project: MPP-Qwen14B & MPP-Qwen-Next(Multimodal Pipeline Parallel based on Qwen-LM). Support [video/image/multi-image] {sft/conversations}. Don't let the poverty limit your imagination! Train your own 8B/14B LLaVA-training-like MLLM on RTX3090/4090 24GB.

R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimization

This project is the official implementation of 'LLMGA: Multimodal Large Language Model based Generation Assistant', ECCV2024 Oral

Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuning

Awesome_Multimodel is a curated GitHub repository that provides a comprehensive collection of resources for Multimodal Large Language Models (MLLM). It covers datasets, tuning techniques, in-context learning, visual reasoning, foundational models, and more. Stay updated with the latest advancement.

[Fully open] [Encoder-free MLLM] Vision as LoRA