Layered, defense-in-depth protection for LLM apps and agents — signatures, multilingual coverage, vector similarity, an ML classifier, canary tokens and agent-trace alignment auditing — fused into one engine with a single API. Honest, reproducible numbers, not marketing.
pip install shadowshield
We publish results on the public suites the field actually cites, not our own. On LLMail-Inject's 2,000 real attack submissions, ShadowShield catches 96.75% at 0% false positives. With the classifier tranche composed in, it drives InjecAgent attack success to 0.1% — and AgentDojo ASR to 0–1.8% across all four suites (baselines 27–62%). Over-defense is the field's failure mode; we measure against it on purpose.
Industry matrix measured 2026-08-09, balanced mode, Wilson 95% CIs. LLMail-Inject: 96.75% catch / 0% FPR (block-only posture: 44.9% / 0% FPR — the gap is attacks sanitized, not missed). InjecAgent: 1,054 cases × 4 arms, gpt-4o-mini, enhanced setting; the sanitize-clf arm (segment-span classifier sanitization) holds 76.9% utility at 0.1% ASR — data-stealing ASR 27.0% → 0.0%. AgentDojo (banking/travel/slack/workspace, classifier arm): ASR 0–1.8% from 27–62% baselines, with no abort-driven utility loss on any suite. *0% false-positive rate on the deepset test split, including NotInject-style hard negatives (6.8% flag-rate, 0% block-FPR, n=339). Frozen blind semantic snapshots are deliberately harder: v1 reaches 26.7% recall / 13.3% FPR, v2 reaches 0% / 10%, and v3 reaches 30% / 30% (v1–v3 aggregate: 22.2% / 20%). † A bundled offline benchmark scores 100% — but that's an in-distribution regression baseline, not a SOTA claim. Full methodology & reproduction in docs/INDUSTRY_BENCHMARKS.md and docs/BENCHMARKS.md.
Eleven detectors and four responders behind one API. Every pane is see-through by design — you can read exactly what fired and why.
Instruction-override, jailbreak, delimiter and exfiltration signatures — matched through zero-width, homoglyph, bidi and base64 normalization so evasions don't slip past.
Embedding similarity to a self-hardening attack corpus catches paraphrases & translations; an opt-in DeBERTa classifier recovers real-world recall.
Canary tokens prove a successful leak; the alignment auditor flags when an action drifts from the user's objective — goal-hijack detection, not just text.
Two-way scanning stops API keys, private keys and PII leaving in model output — Luhn-validated cards, optional Presidio backend. A jailbroken model is still caught at the exit.
Active defense, not just detection: redact the dangerous span, block with a safe fallback, throttle abusers, or spotlight untrusted text so the model can't be steered.
Drop-in for OpenAI-compatible clients & LangChain, an async API, an HTTP server, and an AgentDojo defense adapter. Three modes; YAML config; a plugin system.
Guard input on the way in, scan output on the way out — the same engine, both directions.
import shadowshield as ss shield = ss.Shield.for_mode("strict") # fail-closed: raises on a block clean = shield.guard(user_prompt) reply = my_llm(clean) # two-way: catch leaks on the way out safe = shield.guard(reply, direction="output")
# guard untrusted tool output (indirect injection) shield.scan_tool_result("fetch_url", page_html) # canary: detect a *successful* exfiltration mark = shield.issue_canary() if shield.scan_output(reply).blocked: handle_breach() # goal-hijack auditing across the trace with shield.session(objective=task) as s: s.scan_output(model_action)
"Ignore all previous instructions", new-instruction injection, authority spoofing — in 5 languages at the signature tier.
DAN-style personas, "developer mode", restriction-removal and fiction-wrapper laundering.
Poisoned web pages and tool results that try to steer the agent — scanned as untrusted input.
Actions that drift from the user's stated objective, audited across the execution trace.
Zero-width splits, homoglyphs, bidi overrides and base64/hex payloads — decoded and judged on meaning.
API/private keys, tokens, emails, SSNs and Luhn-valid cards leaving in model output — never echoed to logs.
System-prompt extraction, markdown-image beacons, pipe-to-shell and canary-token leaks.
Adaptive per-identity rate limiting and oversized-input guards built into the request path.
| Capability | LLM Guard | LlamaFirewall | Rebuff | ShadowShield |
|---|---|---|---|---|
| Input + output scanning | ✓ | ✓ | partial | ✓ |
| Multilingual signatures | — | — | — | ✓ |
| Canary tokens | — | — | ✓* | ✓ |
| Agent-trace alignment audit | — | ✓ | — | ✓ |
| Desktop MCP guard server | — | — | — | ✓ |
| Spotlighting as a response | — | — | — | ✓ |
| Self-hardening vector tier | — | — | ✓* | ✓ |
| Published external benchmark | partial | ✓ | — | ✓ |
| License | MIT | Meta | archived | MIT |
*Rebuff pioneered canary tokens & the self-hardening loop, but was archived in 2025 — ShadowShield carries those ideas forward, maintained. Full matrix in docs/COMPARISON.md.
Desktop agents spawn MCP tools all day — each call and each untrusted result is an injection surface. ShadowShield ships a stdio MCP server that guards both, in front of whatever tools you already run.
{
"mcpServers": {
"shadowshield": {
"command": "python",
"args": ["shadowshield_mcp_server.py", "--mode", "balanced"]
}
}
}
from shadowshield.integrations.mcp import build_mcp_server # exposes guard_tool_call / guard_tool_result server = build_mcp_server( ss.Shield.for_mode("balanced"), on_activity=watchdog.ping, # idle-timeout hook ) server.run() # stdio; clients respawn on demand
Tool results are judged, never echoed — verdicts carry detector names and decisions, not payloads.
Speaks both mcp 1.x (FastMCP) and 2.x (MCPServer) — one integration, no pin dance.
An idle-timeout watchdog reaps stale sessions; desktop clients respawn stdio servers on demand.
MIT-licensed. No telemetry. Lightweight by default — the heavy ML is opt-in.