Popular GitHub Repositories for Accuracy Metrics

? Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with OpenTelemetry, Langchain, OpenAI SDK, LiteLLM, and more. ?YC W23

State-of-the-Art Deep Learning scripts organized by models - easy to train and deploy with reproducible accuracy and performance on enterprise-grade infrastructure.

The LLM Evaluation Framework

High accuracy RAG for answering questions from scientific documents with citations

Evidently is ??an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.

Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including OpenAI Agents SDK, CrewAI, Langchain, Autogen, AG2, and CamelAI

High-level library to help with training and evaluating neural networks in PyTorch flexibly and transparently.

《大模型白盒子构建指南》：一个全手搓的Tiny-Universe

mean Average Precision - This code evaluates the performance of your neural net for object recognition.

A comprehensive set of fairness metrics for datasets and machine learning models, explanations for these metrics, and algorithms to mitigate bias in datasets and models.

Machine learning metrics for distributed, scalable PyTorch applications.

Basic Utilities for PyTorch Natural Language Processing (NLP)

Quantized Attention that achieves speedups of 2.1-3.1x and 2.7-5.1x compared to FlashAttention2 and xformers, respectively, without lossing end-to-end metrics across various models.

Avalanche: an End-to-End Library for Continual Learning based on PyTorch.

A minimal benchmark for scalability, speed and accuracy of commonly used open source implementations (R packages, Python scikit-learn, H2O, xgboost, Spark MLlib etc.) of the top machine learning algorithms for binary classification (random forests, gradient boosted trees, deep neural networks etc.).

Neural network graphs and training metrics for PyTorch, Tensorflow, and Keras.

pip install antialiased-cnns to improve stability and accuracy

Powerful Analytics Solution. Setup in 30 seconds. Display all your data on a Simple, AI-powered dashboard. Fully self-hostable and GDPR compliant. Alternative to Google Analytics, MixPanel, Plausible, Umami & Matomo.

Sharing both practical insights and theoretical knowledge about LLM evaluation that we gathered while managing the Open LLM Leaderboard and designing lighteval!

Measures and metrics for image2image tasks. PyTorch.