AI Agent Security Solutions

AI agent security testing and red team exercises

Test an agent security control by giving the agent a harmless task that the control should stop, in a test environment, and confirming the block, the log and the alert. Run exercises per layer, use public technique catalogs to choose cases, and record every gap with an owner.

Facts read on .

How do you test an agent security control?

Pick one control, define the action it should stop, run a harmless version of that action in a test setup and check three things: it was blocked, it was logged and someone was told. If any of the three is missing, the control is not working.

Use test accounts, test data and a sandbox. Never test with production credentials.

Write the expected result before the test. A block test has three pass conditions, and a test with no stated expectation proves nothing. Keep each case short enough that another engineer can repeat it.

Sources: NIST AI Risk Management Framework.

Which attacks should an exercise include?

Include the risk classes NIST and OWASP name: manipulated inputs, excessive agency, credential misuse, risky tools and data leaving. Write each as a scenario, not a script.

NIST's request for information groups risks as adversarial data, insecure models and harmful actions without attack. MITRE ATLAS lists 221 techniques and 77 case studies to pick from.

Prefer realistic scenarios over exotic ones. Most losses start with an over-privileged token or an unreviewed tool, so test those first, then widen to manipulated inputs. Record the layer where each scenario should be stopped.

Sources: NIST CAISI request for information on securing AI agent systems; MITRE ATLAS; OWASP LLM06:2025 Excessive Agency.

What do you test at each layer?

Test the point where the layer acts. On the endpoint, test that a pre-tool check blocks an out-of-scope file read. In the browser, test that an upload to an unapproved AI site is stopped.

In the cloud workspace, test the platform guardrail. In identity, test that a revoked token fails. At the gateway, test that an unlisted tool is denied.

Test the failure modes too. Unplug the network from a test laptop and see whether the endpoint control still decides. Make the gateway unreachable and see whether calls fail closed or open.

Sources: Claude Code hooks reference; Amazon Bedrock: associate a guardrail with your agent; WitnessAI: agentic control.

What tools exist for red teaming agents?

A few are public. Operant publishes Woodpecker, an open-source red teaming tool, on GitHub under Apache-2.0.

Check the license and run any tool only in a test environment. Vendor products often include their own test modes; ask for the documentation.

When you run a vendor's own test mode, ask which actions it covers and which it leaves out. A vendor demo shows the cases the product handles, and your suite should include the ones it does not.

Sources: Operant AI Woodpecker on GitHub.

How do you handle findings?

Log each finding with the layer, the control, the gap, the severity and an owner. Fix the control, not the test.

Retest after the fix and add the case to a standing suite that runs after every policy change.

Share results with the vendor if a control fails. Vendors fix what they hear about, and a documented gap helps your next renewal conversation.

Sources: NIST AI Risk Management Framework.

How often should you run exercises?

Run a small suite monthly and a full exercise twice a year, or after any new agent platform. Agents and their tools change often, so a control that passed last quarter can fail today.

Record each run in the monthly report.

Include a purple-team step: the people who run the test sit with the people who run detection and walk through what each saw. Gaps in what the logs show are often the most useful finding.

Sources: MITRE ATLAS.

Sources