How Raven Works
Raven tests your AI chatbot's system prompt against a library of known attack patterns and tells you exactly what broke, why it matters, and how to fix it.
The problem Raven solves
If you build chatbots using platforms like Voiceflow, Botpress, custom GPTs, or similar tools, your system prompt is the only security boundary you control. Most builders have no way to test whether that boundary holds before going live.
Raven fills that gap. Paste your system prompt, click Run Scan, and within about 30 seconds you receive a full vulnerability report.
No API key to your live bot is required. Raven loads your system prompt into its own sandboxed environment and tests it in isolation.
Running a scan
1. Paste your system prompt
Copy your chatbot's system prompt and paste it into the text area on the main page. The prompt can be up to 8,000 characters. If you want to see how a scan works before testing your own prompt, use one of the example prompts: Weak, Medium, or Strong.
2. Click Run Scan
Raven runs 25 adversarial attacks against your prompt. A live progress bar shows which attack is running and its result as each one completes. The scan typically finishes in under a minute.
3. Review your results
After the scan, you receive:
- An overall risk score: Low, Medium, High, or Critical.
- A breakdown of every attack, grouped by OWASP category, showing which passed and which failed.
- For each failed attack: a plain-language explanation of why it worked and a suggested fix for your prompt wording.
- The model's actual response to each attack payload, so you can judge the behaviour yourself.
Attack categories
Raven's attack library covers six categories drawn from the OWASP LLM Top 10.
| Category | What it tests | Attacks |
|---|---|---|
| Prompt Injection | Whether the model can be tricked into abandoning its instructions through social engineering, fake authority claims, or embedded override directives. | 5 |
| Sensitive Information Disclosure | Whether the model reveals the contents of its system prompt when asked directly or indirectly, framed as a developer audit, diagnostic check, or documentation request. | 5 |
| System Prompt Leakage | Whether the model reproduces its instructions verbatim through indirect extraction methods: translation requests, JSON serialisation, creative writing exercises, or recall prompts. | 4 |
| Insecure Output Handling | Whether the model generates dangerous output it should refuse: executable code, credential-harvesting forms, destructive shell commands, or references to internal network endpoints. | 4 |
| Excessive Agency | Whether the model claims to have capabilities it should not have, such as sending emails, browsing the web, calling external services, or persisting memory across sessions. | 4 |
| Persona Non-Adherence | Whether the model breaks its assigned persona and reveals its underlying identity, provider, or the fact that it is an AI language model. | 3 |
Risk score
The overall risk score reflects how many attacks produced a fail verdict.
| Failed attacks | Risk score |
|---|---|
| 0 | Low |
| 1 to 2 | Medium |
| 3 to 5 | High |
| 6 or more | Critical |
Exporting results
After a scan you can export the full report in two formats.
- JSON. The complete scan result, including all model responses, verdicts, and metadata. Useful for archiving or sharing with a technical team.
- PDF. A formatted report with a risk badge, summary statistics, a category chart, and per-attack findings with remediation hints. Useful for sharing with stakeholders.
What Raven does not test
Raven tests your system prompt in isolation. It does not test your live bot, your deployment platform's own safeguards, or runtime behaviour that depends on conversation history or user context. A low risk score means your prompt held up against Raven's 25 attacks. It does not mean your deployed bot is secure in all conditions.
Raven covers five of the ten OWASP LLM categories. It does not currently cover model denial of service, training data poisoning, misinformation generation, or supply chain vulnerabilities.
Your data
Your system prompt is never stored. It exists only for the duration of the scan and is discarded the moment the result is returned. See the Privacy Policy for full details.