Popular GitHub Repositories for Jailbreak Attacks

ChatGPT DAN, Jailbreaks prompt

List of free GPTs that doesn't require plus subscription

An adversarial example library for constructing attacks, building defenses, and benchmarking both

GPT 3.5/4 with a Chat Web UI. No API key required.

Adversarial Robustness Toolbox (ART) - Python Library for Machine Learning Security - Evasion, Poisoning, Extraction, Inference - Red and Blue Teams

Data augmentation for NLP

[CCS'24] A dataset consists of 15,140 ChatGPT prompts from Reddit, Discord, websites, and open-source datasets (including 1,405 jailbreak prompts).

TextAttack ? is a Python framework for adversarial attacks, data augmentation, and model training in NLP https://textattack.readthedocs.io/en/master/

A Python toolbox to create adversarial examples that fool neural networks in PyTorch, TensorFlow, and JAX

ChatGPT Jailbreaks, GPT Assistants Prompt Leaks, GPTs Prompt Injection, LLM Prompt Security, Super Prompts, Prompt Hack, Prompt Security, Ai Prompt Engineering, Adversarial Machine Learning.

A unified evaluation framework for large language models

UpTrain is an open-source unified platform to evaluate and improve Generative AI applications. We provide grades for 20+ preconfigured checks (covering language, code, embedding use-cases), perform root cause analysis on failure cases and give insights on how to resolve them.

PyTorch implementation of adversarial attacks [torchattacks]

A reading list for large models safety, security, and privacy (including Awesome LLM Security, Safety, etc.).

Must-read Papers on Textual Adversarial Attack and Defense

Advbox is a toolbox to generate adversarial examples that fool neural networks in PaddlePaddle、PyTorch、Caffe2、MxNet、Keras、TensorFlow and Advbox can benchmark the robustness of machine learning models. Advbox give a command line tool to generate adversarial examples with Zero-Coding.

A Toolbox for Adversarial Robustness Research

A pytorch adversarial library for attack and defense methods on images and graphs

Awesome things you can do with ChatGPT + Code Interpreter combo ?

A cross-platform desktop client for the jailbroken New Bing AI Copilot (Sydney ver.) built with Go and Wails (previously based on Python and Qt).