Popular GitHub Repositories for Nats Bench

SWE-bench [Multimodal]: Can Language Models Resolve Real-world Github Issues?

High performance Rust stream processing engine seamlessly integrates AI capabilities, providing powerful real-time data processing and intelligent analysis.

The 100 line AI agent that solves GitHub issues or helps you in your command line. Radically simple, no crazy configs, no giant monorepo—but scores 65% on SWE-bench verified!

Z-Bench 1.0 by 真格基金：一个麻瓜的大语言模型中文测试集。Z-Bench is a LLM prompt dataset for non-technical users, developed by an enthusiastic AI-focused team in Zhenfund.

Codes for the paper "∞Bench: Extending Long Context Evaluation Beyond 100K Tokens": https://arxiv.org/abs/2402.13718

The Official Repo of ML-Bench: Evaluating Large Language Models and Agents for Machine Learning Tasks on Repository-Level Code (https://arxiv.org/abs/2311.09835)

DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents

[NeurIPS 2024 D&B Spotlight?] ChronoMagic-Bench: A Benchmark for Metamorphic Evaluation of Text-to-Time-lapse Video Generation

[ICLR 2021] "Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective" by Wuyang Chen, Xinyu Gong, Zhangyang Wang

CodeRAG-Bench: Can Retrieval Augment Code Generation?

Isaaclab-based grasp learning test bench

Official repository for our paper "FullStack Bench: Evaluating LLMs as Full Stack Coders"

The D-CIPHER and NYU CTF baseline LLM Agents built for NYU CTF Bench

CVE-Bench: A Benchmark for AI Agents’ Ability to Exploit Real-World Web Application Vulnerabilities

CodeUltraFeedback: aligning large language models to coding preferences

Core microservices powering the DanteGPU distributed GPU network. Manages providers, orchestrates AI job scheduling, handles auth, data, and monitoring. Built with Go & Python.

Aix-bench, the Java benchmark for code synthesis problem.

[CVPR 2025 ?] ALM-Bench is a multilingual multi-modal diverse cultural benchmark for 100 languages across 19 categories. It assesses the next generation of LMMs on cultural inclusitivity.

LLM-KG-Bench is a Framework and task collection for automated benchmarking of Large Language Models (LLMs) on Knowledge Graph (KG) related tasks.

the small distributed language model toolkit; fine-tune state-of-the-art LLMs anywhere, rapidly