LLM Red-Teaming Evaluation Pipeline

An extensible framework for measuring how well language models resist prompt-injection attacks—and where their defenses quietly break down.

The Problem

As LLMs get wired into agentic systems with tool access, prompt injection becomes a real security surface rather than a curiosity. But "is this model safe?" is not a yes/no question—robustness varies by attack technique, by model size, and by the defenses layered on top. I set out to build a rigorous, repeatable way to quantify that robustness instead of reasoning about it anecdotally.

Architecture & Implementation

I designed an extensible prompt-injection evaluation framework with cleanly separated, pluggable layers—provider, target, classifier, and defense—so new models, attacks, and mitigations can be swapped in without touching the core harness.

• Attack Corpus: 46 attack payloads spanning 4 categories and 41 distinct techniques, exercised against both local and frontier models.
• Dual Scoring: A hybrid scoring pipeline combining deterministic rule-based checks with an LLM-as-judge, surfacing breakthroughs a single method would miss.
• Dashboard & CI: A Streamlit dashboard for exploring results, backed by 35 automated tests running in CI.

Key Findings

The pipeline quantified a robustness gap of up to a 47% breakthrough rate on local 8B models versus near-zero on a frontier model—a stark, measurable difference in resilience. Through multi-model, repeated-trial experiments, it also surfaced non-obvious failure modes: prompt-hardening defenses that merely relocated vulnerabilities rather than removing them, and denylist filters that were defeated by trivial obfuscation. The takeaway: defenses that look effective in isolation can give a false sense of security until they're measured systematically.