Popular GitHub Repositories for Safety

A curated list of awesome responsible machine learning resources.

An open-source data logging library for machine learning models and data pipelines. ? Provides visibility into data quality & model performance over time. ?️ Supports privacy-preserving data collection, ensuring safety & robustness. ?

A reading list for large models safety, security, and privacy (including Awesome LLM Security, Safety, etc.).

Safe RLHF: Constrained Value Alignment via Safe Reinforcement Learning from Human Feedback

Secrets of RLHF in Large Language Models Part I: PPO

Corruption and Perturbation Robustness (ICLR 2019)

Chinese safety prompts for evaluating and improving the safety of LLMs. 中文安全prompts，用于评估和提升大模型的安全性。

JMLR: OmniSafe is an infrastructural framework for accelerating SafeRL research.

? LangKit: An open-source toolkit for monitoring Large Language Models (LLMs). ? Extracts signals from prompts & responses, ensuring safety & security. ?? Features include text quality, relevance metrics, & sentiment analysis. ? A comprehensive tool for LLM observability. ?

UQLM: Uncertainty Quantification for Language Models, is a Python package for UQ-based LLM hallucination detection

Awesome-Jailbreak-on-LLMs is a collection of state-of-the-art, novel, exciting jailbreak methods on LLMs. It contains papers, codes, datasets, evaluations, and analyses.

Awesome-LLM-Robustness: a curated list of Uncertainty, Reliability and Robustness in Large Language Models

PyBullet CartPole and Quadrotor environments—with CasADi symbolic a priori dynamics—for learning-based control and RL

Discover top AI coding tools for developers, including code editors, completion engines, and natural language converters. Enhance your workflow with AI! ?️?

The repository is for safe reinforcement learning baselines.

A framework to evaluate the generalization capability of safety alignment for LLMs

[CoRL 2022] InterFuser: Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformer

Deep Anomaly Detection with Outlier Exposure (ICLR 2019)

Deliver safe & effective language models

面向中文大模型价值观的评估与对齐研究