Popular GitHub Repositories for Codal Bench

SWE-bench [Multimodal]: Can Language Models Resolve Real-world Github Issues?

The 100 line AI agent that solves GitHub issues or helps you in your command line. Radically simple, no crazy configs, no giant monorepo—but scores 65% on SWE-bench verified!

Z-Bench 1.0 by 真格基金：一个麻瓜的大语言模型中文测试集。Z-Bench is a LLM prompt dataset for non-technical users, developed by an enthusiastic AI-focused team in Zhenfund.

DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents

Codes for the paper "∞Bench: Extending Long Context Evaluation Beyond 100K Tokens": https://arxiv.org/abs/2402.13718

The Official Repo of ML-Bench: Evaluating Large Language Models and Agents for Machine Learning Tasks on Repository-Level Code (https://arxiv.org/abs/2311.09835)

[NeurIPS 2024 D&B Spotlight?] ChronoMagic-Bench: A Benchmark for Metamorphic Evaluation of Text-to-Time-lapse Video Generation

[ICLR 2021] "Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective" by Wuyang Chen, Xinyu Gong, Zhangyang Wang

SE-Agent is a self-evolution framework that enables information exchange between reasoning paths through a trajectory-level evolution mechanism, breaking the cognitive limitations of single trajectories.

CodeRAG-Bench: Can Retrieval Augment Code Generation?

Isaaclab-based grasp learning test bench

Official repository for our paper "FullStack Bench: Evaluating LLMs as Full Stack Coders"

CVE-Bench: A Benchmark for AI Agents’ Ability to Exploit Real-World Web Application Vulnerabilities

The D-CIPHER and NYU CTF baseline LLM Agents built for NYU CTF Bench

CodeUltraFeedback: aligning large language models to coding preferences

Aix-bench, the Java benchmark for code synthesis problem.

[CVPR 2025 ?] ALM-Bench is a multilingual multi-modal diverse cultural benchmark for 100 languages across 19 categories. It assesses the next generation of LMMs on cultural inclusitivity.

LLM-KG-Bench is a Framework and task collection for automated benchmarking of Large Language Models (LLMs) on Knowledge Graph (KG) related tasks.

[AAAI '23] PINAT: A Permutation INvariance Augmented Transformer for NAS Predictor

[NeurIPS 2021] “Stronger NAS with Weaker Predictors“, Junru Wu, Xiyang Dai, Dongdong Chen, Yinpeng Chen, Mengchen Liu, Ye Yu, Zhangyang Wang, Zicheng Liu, Mei Chen and Lu Yuan