Popular GitHub Repositories for Purple Teaming

所有系统提示都需透明公开

Test your prompts, agents, and RAGs. Red teaming, pentesting, and vulnerability scanning for LLMs. Compare performance of GPT, Claude, Gemini, Llama, and more. Simple declarative configs with command line and CI/CD integration.

Agentic LLM Vulnerability Scanner / AI red teaming kit ?

A security scanner for your LLM agentic workflows

Automated prompt-based testing and evaluation of Gen AI applications

[NeurIPS 2024] Official implementation for "AgentPoison: Red-teaming LLM Agents via Memory or Knowledge Base Backdoor Poisoning"

Framework for testing vulnerabilities of large language models (LLM).

AI Alignment: A Comprehensive Survey

Papers about red teaming LLMs and Multimodal models.

Whistleblower is a offensive security tool for testing against system prompt leakage and capability discovery of an AI application exposed through API. Built for AI engineers, security researchers and folks who want to know what's going on inside the LLM-based app they use daily

Blackdagger is a DAG-based automation tool specifically used in DevOps, DevSecOps, MLOps, MLSecOps, and Continuous Red Teaming (CART).

Codes and datasets of the paper Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignment

Code scanner to check for issues in prompts and LLM calls

Code Repository for: AIRTBench: Measuring Autonomous AI Red Teaming Capabilities in Language Models

BeaconatorC2 is a framework for red teaming and adversarial emulation, providing a full-featured management interface, along with a catalog of beacons and a clear schema to add more beacons over time.

PromptMe is an educational project that showcases security vulnerabilities in large language models (LLMs) and their web integrations. It includes 10 hands-on challenges inspired by the OWASP LLM Top 10, demonstrating how these vulnerabilities can be discovered and exploited in real-world scenarios.

A Completely Modular LLM Reverse Engineering, Red Teaming, and Vulnerability Research Framework.

Official repository for the paper "ALERT: A Comprehensive Benchmark for Assessing Large Language Models’ Safety through Red Teaming"

A re-implementation of the "Red Teaming Language Models with Language Models" paper by Perez et al., 2022

Risks and targets for assessing LLMs & LLM vulnerabilities