Stress-test agent defenses with AgentDojo
Run AgentDojo benchmark environments to evaluate prompt-injection attacks and defenses against LLM agents before trusting them with real tools or data.
npx skills add agentskillexchange/skills --skill stress-test-agent-defenses-with-agentdojo
Use AgentDojo when an agent needs a reproducible security evaluation pass against prompt-injection and tool-use failures. The operator chooses an AgentDojo task environment, wires the target model or agent pipeline, runs attack and defense scenarios, inspects benchmark failures, and turns the results into concrete hardening work before deployment.
What this skill actually does
Invoke this instead of normal product testing when the risk is adversarial instructions, compromised tool outputs, or agent policies that look fine on happy-path prompts but fail under attack. Keep the scope to benchmarked attack and defense evaluation for LLM agents. Do not use it as a general security scanner, a jailbreak prompt list, or a generic agent framework.
Inputs and prerequisites: Python 3.10+, agentdojo package, model provider credentials for the target pipeline, optional transformers extra for prompt-injection detector workflows.
Setup notes: Install with `pip install agentdojo`. For detector workflows, install `pip install “agentdojo[transformers]”`. Then follow the upstream benchmark documentation to configure the target model or agent pipeline.
Source and verification boundary: use https://agentdojo.spylab.ai/ as the canonical reference before running the workflow; keep commands, API calls, CLI usage, and generated outputs reviewable against that upstream source.
Framework fit: publish this as a Multi-Framework workflow only when the operator can invoke the documented toolchain directly, rather than treating the upstream project as a generic product listing.