Popular GitHub Repositories for Inter Modal

MiniCPM-o 2.6: A GPT-4o Level MLLM for Vision, Speech and Multimodal Live Streaming on Your Phone

Awesome pre-trained models toolkit based on PaddlePaddle. (400+ models including Image, Text, Audio, Video and Cross-Modal with Easy Inference & Serving)

? Scalable embedding, reasoning, ranking for images and sentences with CLIP

An open source implementation of CLIP.

Database for AI. Store Vectors, Images, Texts, Videos, etc. Use with LLMs/LangChain. Store, query, version, & visualize any AI data. Stream data in real-time to PyTorch/TensorFlow. https://activeloop.ai

[CVPR 2024 Oral] InternVL Family: A Pioneering Open-Source Alternative to GPT-4o. 接近GPT-4o表现的开源多模态对话模型

ModelScope: bring the notion of Model-as-a-Service to life.

a state-of-the-art-level open visual language model | 多模态预训练模型

Implementation of all RAG techniques in a simpler way

MMdnn is a set of tools to help users inter-operate among different deep learning frameworks. E.g. model conversion and visualization. Convert models between Caffe, Keras, MXNet, Tensorflow, CNTK, PyTorch Onnx and CoreML.

Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch

Chinese version of CLIP which achieves Chinese cross-modal retrieval and representation generation.

The Enterprise-Grade Production-Ready Multi-Agent Orchestration Framework. Website: https://swarms.ai

Unified embedding generation and search engine. Also available on cloud - cloud.marqo.ai

Lord of Large Language and Multi modal Systems Web User Interface

Data processing for and with foundation models! ? ? ? ?? ??? ? ?

Transformers for Information Retrieval, Text Classification, NER, QA, Language Modelling, Language Generation, T5, Multi-Modal, and Conversational AI

OmniGen: Unified Image Generation. https://arxiv.org/pdf/2409.11340

Chinese and English multimodal conversational language model | 多模态中英双语对话语言模型

[EMNLP 2022] An Open Toolkit for Knowledge Graph Extraction and Construction