Popular GitHub Repositories for Evaluation Datasets

Repository to track the progress in Natural Language Processing (NLP), including the datasets and the current state-of-the-art for the most common NLP tasks.

? The largest hub of ready-to-use datasets for ML models with fast, easy-to-use and efficient data manipulation tools

Library of deep learning models and datasets designed to make deep learning more accessible and accelerate ML research.

Python SDK for Agent AI Observability, Monitoring and Evaluation Framework. Includes features like agent, llm and tools tracing, debugging multi-agentic system, self-hosted dashboard and advanced analytics with timeline and execution graph view

? Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with OpenTelemetry, Langchain, OpenAI SDK, LiteLLM, and more. ?YC W23

Transfer learning / domain adaptation / domain generalization / multi-task learning etc. Papers, codes, datasets, applications, tutorials.-迁移学习

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

pix2code: Generating Code from a Graphical User Interface Screenshot

The standard data-centric AI package for data quality and machine learning with messy, real-world data and labels.

Supercharge Your LLM Application Evaluations ?

Open source annotation tool for machine learning practitioners.

The LLM Evaluation Framework

A powerful tool for creating fine-tuning datasets for LLM

A framework for few-shot evaluation of language models.

Refine high-quality datasets and visual AI models

Techniques for deep learning with satellite & aerial imagery

BISHENG is an open LLM devops platform for next generation Enterprise AI applications. Powerful and comprehensive features include: GenAI workflow, RAG, Agent, Unified model management, Evaluation, SFT, Dataset Management, Enterprise-level System Management, Observability and more.

Database for AI. Store Vectors, Images, Texts, Videos, etc. Use with LLMs/LangChain. Store, query, version, & visualize any AI data. Stream data in real-time to PyTorch/TensorFlow. https://activeloop.ai

BELLE: Be Everyone's Large Language model Engine（开源中文对话大模型）

Test your prompts, agents, and RAGs. Red teaming, pentesting, and vulnerability scanning for LLMs. Compare performance of GPT, Claude, Gemini, Llama, and more. Simple declarative configs with command line and CI/CD integration.