PromptInject
PromptInject is a framework that assembles prompts in a modular fashion to provide a quantitative analysis of the robustness of LLMs to adversarial prompt attac
What is PromptInject?
PromptInject is a framework that assembles prompts in a modular fashion to provide a quantitative analysis of the robustness of LLMs to adversarial prompt attacks. π Best Paper Awards @ NeurIPS ML Safety Workshop 2022
Overview of PromptInject
What it is
PromptInject is a framework that assembles prompts in a modular fashion to provide a quantitative analysis of the robustness of large language models (LLMs) to adversarial prompt attacks. It was presented in the paper 'Ignore Previous Prompt: Attack Techniques For Language Models' and received the Best Paper Award at the NeurIPS ML Safety Workshop 2022. The framework focuses on two types of attacks: goal hijacking and prompt leaking. It uses a mask-based iterative adversarial prompt composition approach to examine how models like GPT-3 can be misaligned by handcrafted inputs. The project is open source and hosted on GitHub under the MIT license.
Key features
- Modular prompt assembly for constructing adversarial inputs.
- Quantitative analysis of LLM robustness to adversarial prompt attacks.
- Support for two attack types: goal hijacking (changing the model's output to a specific target string) and prompt leaking (extracting the original application prompt).
- Mask-based iterative adversarial prompt composition.
- Designed to evaluate models such as GPT-3, as demonstrated in the associated paper.
Use cases
- Security research on large language models.
- Evaluating the robustness of customer-facing LLM applications.
- Studying the vulnerabilities of transformer-based models to malicious user inputs.
- Developing defenses against adversarial prompt attacks.
Getting started
Install the package via pip: pip install git+https://github.com/agencyenterprise/PromptInject For usage examples, see the notebook notebooks/Example.ipynb in the repository.
License and pricing
The project is licensed under the MIT License. It is open source and free to use.
Based on the project's public README and repository metadata.