WavJEPA is an audio foundation model based on the waveform joint embedding prediction architecture. It uses advanced semantic representation learning to address the deficiencies in speech unit or token-level representation learning. It significantly outperforms the state-of-the-art time-domain audio foundation models in many downstream benchmark tasks while requiring significantly less computational resources.

Audio ProcessingTransformers

CoDA is a code generation language model based on diffusion models developed by Salesforce AI Research. It has the ability to understand bidirectional context and is specifically designed for powerful code generation and completion tasks. This model only has 1.7 billion parameters and achieves excellent code generation performance while maintaining low computational requirements.

Natural Language ProcessingTransformersEnglish

SRPO is a human preference alignment method for diffusion models. Through Direct-Align technology and semantic relative preference optimization, it significantly improves the realism and aesthetic quality of the FLUX.1-dev model, and solves the problems of high computational cost in multi-step denoising and dependence on offline reward fine-tuning.

Computer VisionDiffusers

GLM-4.5V-AWQ-4bit is a quantized multimodal model built on Zhipu AI's next-generation flagship text foundation model. Optimized by AWQ-4bit quantization technology, it significantly reduces the computational resource requirements while maintaining excellent performance. This model achieves SOTA performance among models of the same scale in 42 public visual language benchmark tests and has powerful visual reasoning capabilities.

MultimodalTransformersMultiple Languages

Falcon-H1 is a new family of large language models developed by the Technology Innovation Institute of the United Arab Emirates. It adopts a hybrid architecture that combines the Transformer attention mechanism and the state space model (SSMs), with excellent long-context memory ability and computational efficiency. This series of models offers various configurations with parameters ranging from 0.5B to 34B, and performs excellently in tasks such as inference, mathematics, and multilingual tasks.

Natural Language ProcessingTransformersMultiple Languages

The first open-source 2-billion-parameter native 1-bit large language model developed by Microsoft Research, trained on 4 trillion tokens, demonstrating that native 1-bit large language models can significantly improve computational efficiency while maintaining performance comparable to full-precision open-source models of the same scale.

Natural Language ProcessingTransformersEnglish

The Trillion-7B Preview is a multilingual large language model supporting English, Korean, Japanese, and Chinese. It outperforms other 7B-scale models in computational efficiency and performance.

Natural Language ProcessingGgufMultiple Languages

The Trillion-7B Preview is a multilingual large language model supporting English, Korean, Japanese, and Chinese. It achieves performance competitive with higher-computation models while maintaining lower computational requirements.

Natural Language ProcessingTransformersMultiple Languages

Kanana is a bilingual (Korean/English) language model series developed by Kakao, excelling in Korean tasks while remaining competitive in English tasks, significantly reducing computational costs compared to models of similar scale.

Natural Language ProcessingTransformersMultiple Languages

Kanana is a bilingual (Korean/English) language model series developed by Kakao. This 2.1B parameter version outperforms similar models in Korean while maintaining efficient computational costs.

Natural Language ProcessingTransformersMultiple Languages

Kokoro is an open-source text-to-speech model with 82 million parameters, achieving sound quality comparable to large models with a lightweight architecture while improving generation speed and reducing computational costs.

The TestSavantAI model is a set of classifiers specifically designed to defend against prompt injection and jailbreak attacks in large language models (LLMs). The tiny version is based on the BERT-tiny architecture, balancing security and computational efficiency.

Natural Language ProcessingTransformersEnglish

This is a collection of large language models quantized to 2 bits using the novel quip# method in llama.cpp, which includes multiple high-performance models such as Senku-70b, Nous-Hermes2-70b, Miquliz-120b-v2.0, etc. These models significantly reduce storage and computational requirements while maintaining high performance.

Natural Language ProcessingGguf

This research explores the most effective activation functions for sparse large language models, comparing the performance of ReLU, SwiGLU, ReGLU, and squared ReLU in terms of sparse computational efficiency.

Natural Language ProcessingTransformersEnglish

LongLoRA is an efficient fine-tuning method that can extend the context length of pre-trained large language models with limited computational cost.

Natural Language ProcessingTransformers

MoLM is a series of language models based on the Mixture of Experts (MoE) architecture. The 700M-4B version has a total of 4 billion parameters, with computational consumption equivalent to a dense model of 700 million parameters.

Natural Language ProcessingTransformers

LongLoRA is an efficient fine-tuning method for large language models that can extend the context length of pre-trained models at a limited computational cost. This method significantly reduces the computational requirements for long-context training while maintaining model performance through shifted short attention and improved LoRA technology.

Natural Language ProcessingTransformers

The Latent Diffusion Model (LDM) is an efficient image generation method that applies diffusion models in latent space, significantly reducing computational requirements while maintaining high-quality generation results.

Computer VisionDiffusers