Popular GitHub Repositories for Jailbreaking

[CCS'24] A dataset consists of 15,140 ChatGPT prompts from Reddit, Discord, websites, and open-source datasets (including 1,405 jailbreak prompts).

This is a repository that aims to provide updates on the status of jailbreaking the OpenAI GPT language model.

A powerful tool for automated LLM fuzzing. It is designed to help developers and security researchers identify and mitigate potential jailbreaks in their LLM APIs.

This repository contains the jailbreaking process for GPT-3, GPT-4, GPT-3.5, ChatGPT, and ChatGPT Plus. By following the instructions in this repository, you will be able to gain access to the inner workings of these language models and modify them to your liking.

[AAAI'25 (Oral)] Jailbreaking Large Vision-language Models via Typographic Visual Prompts

Does Refusal Training in LLMs Generalize to the Past Tense? [ICLR 2025]

Materials for the course Principles of AI: LLMs at UPenn (Stat 9911, Spring 2025). LLM architectures, training paradigms (pre- and post-training, alignment), test-time computation, reasoning, safety and robustness (jailbreaking, oversight, uncertainty), representations, interpretability (circuits), etc.

Open-source LLM Prompt-Injection and Jailbreaking Playground

TurboFuzzLLM: Turbocharging Mutation-based Fuzzing for Effectively Jailbreaking Large Language Models in Practice

This is the official repository for Cross-modality Information Check for Detecting Jailbreaking in Multimodal Large Language Models.

Your best llm security paper library

A flexible and portable solution that uses a single robust prompt and customized hyperparameters to classify user messages as either malicious or safe, helping to prevent jailbreaking and manipulation of chatbots and other LLM-based solutions.

Multi-agent system for query processing with safety verification and critique built with Google A2A protocol, Google ADK, Llama Prompt Guard 2, Gemma 3 and Gemini 2.0 Flash.

Exploring the AntiGPT Prompt: A Deep Dive

This repository contains the codebase for Jailbreaking Deep Models, which investigates the vulnerability of deep convolutional neural networks to adversarial attacks. The project systematically implements and analyzes Fast Gradient Sign Method (FGSM), Projected Gradient Descent (PGD), and localized patch-based attacks on the pretrained