Popular GitHub Repositories for Webperf Metrics

? Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with OpenTelemetry, Langchain, OpenAI SDK, LiteLLM, and more. ?YC W23

The LLM Evaluation Framework

Evidently is ??an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.

Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including OpenAI Agents SDK, CrewAI, Langchain, Autogen, AG2, and CamelAI

High-level library to help with training and evaluating neural networks in PyTorch flexibly and transparently.

《大模型白盒子构建指南》：一个全手搓的Tiny-Universe

mean Average Precision - This code evaluates the performance of your neural net for object recognition.

A comprehensive set of fairness metrics for datasets and machine learning models, explanations for these metrics, and algorithms to mitigate bias in datasets and models.

Machine learning metrics for distributed, scalable PyTorch applications.

Basic Utilities for PyTorch Natural Language Processing (NLP)

Quantized Attention that achieves speedups of 2.1-3.1x and 2.7-5.1x compared to FlashAttention2 and xformers, respectively, without lossing end-to-end metrics across various models.

Avalanche: an End-to-End Library for Continual Learning based on PyTorch.

Neural network graphs and training metrics for PyTorch, Tensorflow, and Keras.

Powerful Analytics Solution. Setup in 30 seconds. Display all your data on a Simple, AI-powered dashboard. Fully self-hostable and GDPR compliant. Alternative to Google Analytics, MixPanel, Plausible, Umami & Matomo.

Measures and metrics for image2image tasks. PyTorch.

Sharing both practical insights and theoretical knowledge about LLM evaluation that we gathered while managing the Open LLM Leaderboard and designing lighteval!

一款多语言的API，满足您所有可观测性需求。轻量、快速，可替代LGTM :rocket: 独立的OpenTelemetry、Loki、Prometheus、Tempo、Pyroscope兼容 :star:

Evaluation code for various unsupervised automated metrics for Natural Language Generation.

Object Detection Metrics. 14 object detection metrics: mean Average Precision (mAP), Average Recall (AR), Spatio-Temporal Tube Average Precision (STT-AP). This project supports different bounding box formats as in COCO, PASCAL, Imagenet, etc.

High-fidelity performance metrics for generative models in PyTorch