Popular GitHub Repositories for Safety Monitoring

Python SDK for Agent AI Observability, Monitoring and Evaluation Framework. Includes features like agent, llm and tools tracing, debugging multi-agentic system, self-hosted dashboard and advanced analytics with timeline and execution graph view

? Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with OpenTelemetry, Langchain, OpenAI SDK, LiteLLM, and more. ?YC W23

Free MLOps course from DataTalks.Club

Build high-quality LLM apps - from prototyping, testing to production deployment and monitoring.

AI Observability & Evaluation

Evidently is ??an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.

Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including OpenAI Agents SDK, CrewAI, Langchain, Autogen, AG2, and CamelAI

Next-generation AI Agent Optimization Platform: Cozeloop addresses challenges in AI agent development by providing full-lifecycle management capabilities from development, debugging, and evaluation to monitoring.

? Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 ?

Stunning status pages, batteries included!

Real-time Claude Code usage monitor with predictions and warnings

Deepchecks: Tests for Continuous Validation of ML Models & Data. Deepchecks is a holistic open-source solution for all of your AI & ML validation needs, enabling to thoroughly test your data and models from research to production.

A curated list of awesome responsible machine learning resources.

CLI tool for configuring and monitoring Claude Code

The open-source AIOps and alert management platform

Debugging, monitoring and visualization for Python Machine Learning and Data Science

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM Observability all in one place.

?️ Open-source, developer-first LLMOps platform designed to streamline prompt design, version management, instant delivery, collaboration, troubleshooting, observability and more.

Better Prometheus alerts for Kubernetes - smart grouping, AI enrichment, and automatic remediation

An open-source data logging library for machine learning models and data pipelines. ? Provides visibility into data quality & model performance over time. ?️ Supports privacy-preserving data collection, ensuring safety & robustness. ?