Stable Diffusion v2-1-unclip is a diffusion model fine-tuned based on Stable Diffusion 2.1. It can accept text prompts and CLIP image embeddings, and is used to create image variants or be used in combination with the text-to-image CLIP prior.

MultimodalDiffusers

MM-VAE Lyra is a multimodal variational autoencoder specifically designed for text embedding conversion, using geometric fusion technology. It combines the CLIP-L and T5-base models and can effectively handle the encoding and decoding tasks of text embeddings, providing an innovative solution for multimodal data processing.

This is a Transformer model published on the Hugging Face model hub. The model card is automatically generated by the system, and specific model information needs to be further supplemented.

Natural Language ProcessingTransformers

This is a ViT-L14 image encoder based on the PE-Core model by Bolya et al., which has been converted to the Birder format for image feature extraction. The model retains the original weights and architecture but removes the CLIP projection layer to output raw image embeddings. It is a general-purpose visual backbone network suitable for image classification and detection tasks.

Computer VisionPytorch

This is an automatically generated 🤗 Transformers model card, lacking specific model information.

Natural Language ProcessingTransformers

DermLIP is a dermatological vision-language model trained on the Derm1M dataset. It uses a CLIP-style contrastive learning method and is specifically optimized for dermatological images and texts, supporting various application scenarios such as zero-shot classification and few-shot learning.

MultimodalTransformersEnglish

DermLIP is a vision-language model specifically designed for the field of dermatology, trained on the largest dermatology image-text corpus, Derm1M. This model adopts a CLIP-style architecture and can perform various dermatology-related tasks, including zero-shot classification, few-shot learning, cross-modal retrieval, and concept annotation.

MultimodalTransformersEnglish

Llama3-MS-CLIP is the first visual language model in the CLIP family that can understand multispectral images. It is trained on one million image-text pairs from the SSL4EO-S12-v1.1 dataset and the generated descriptions, and outperforms other RGB-based models in most benchmark tests.

Based on the OpenCLIP-ViT-L-14 model, the Test-Time Register technology is introduced to improve the model's interpretability and the performance of downstream tasks.

MultimodalTransformers

A vision-language model based on the OpenCLIP-ViT-B-16 architecture. By introducing test-time registers to optimize the internal representation, it solves the problem of feature map artifacts.

MultimodalTransformers

Vision-language model fine-tuned based on CLIP-ViT-B/32, suitable for image-text matching tasks

MultimodalTransformers

OpenVision is a fully open-source, cost-effective advanced visual encoder family designed for multimodal learning, with performance matching or surpassing OpenAI CLIP.

MultimodalTransformers

A vision-language model fine-tuned based on the CLIP architecture, specifically designed for zero-shot classification of plant diseases

Computer VisionSafetensorsEnglish

Table LLaVA 7B is an open-source multimodal chatbot specifically designed to understand table images and can perform various table-related tasks such as table question answering, table cell description, and structure understanding. This model is based on the LLaVA-v1.5 architecture, using CLIP-ViT-L-336px as the visual encoder and Vicuna-v1.5-7B as the base large language model.

MultimodalSafetensorsEnglish

A feature extraction model fine-tuned based on openai/clip-vit-large-patch14, optimizing the image and text encoders

MultimodalTransformers

This is a speaker segmentation model based on powerset encoding, capable of processing 10-second audio clips and identifying multiple speakers and their overlapping situations.

Audio ProcessingPytorch

PHOENIX is a domain-adaptive model based on CLIP/ViT, designed to enhance patent image retrieval capabilities, particularly suitable for retrieving semantically or hierarchically related images rather than exact matches.

Computer VisionSafetensors

viⓍTTS is a voice generation model capable of cloning voices into different languages using a 6-second short audio clip.

This is a fine-tuned version of CLIP-GPT2 for real-time image captioning tasks, designed to assist visually impaired individuals in understanding image content.

MultimodalTransformers

SciScore is a fine-tuned scientific scoring model based on the CLIP-H model, used to evaluate the scientific alignment between implicit prompts and generated images.

MultimodalTransformers