NAS-Bench-Macro

? A ranked list of awesome machine learning Python libraries. Updated weekly.

Python SDK for Agent AI Observability, Monitoring and Evaluation Framework. Includes features like agent, llm and tools tracing, debugging multi-agentic system, self-hosted dashboard and advanced analytics with timeline and execution graph view

An open source AutoML toolkit for automate machine learning lifecycle, including feature engineering, neural architecture search, model compression and hyper-parameter tuning.

? Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with OpenTelemetry, Langchain, OpenAI SDK, LiteLLM, and more. ?YC W23

A MNIST-like fashion product database. Benchmark :point_down:

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

Supercharge Your LLM Application Evaluations ?

A Python Automated Machine Learning tool that optimizes machine learning pipelines using genetic programming.

A framework for few-shot evaluation of language models.

The LLM Evaluation Framework