RAMPART

RAMPART (Risk Assessment & Measurement Platform for Agentic Red Teaming) is an open-source framework from Microsoft for writing and running safety and security tests against AI agents. Tests are written with pytest, and RAMPART handles the interaction with the agent, evaluates the outcome, and reports results. The documentation marks the project as alpha, with APIs that may change before 1.0; the latest release listed is v0.1.0.

What It Is

RAMPART is a testing framework for agentic AI applications. Developers write tests that attack or probe their agent, covering adversarial attacks, benign failures, and a range of harm categories, with evaluation-driven assertions. The agent is connected to the framework through an adapter that the user provides.

How It Works

RAMPART supplies execution strategies that orchestrate injection, triggering, and evaluation lifecycles. Evaluators detect conditions in agent responses such as tool calls, text patterns, and side effects. The docs describe an XPIA attack type and a behavioral probe type, plus a PyRIT integration.

pytest and CI Workflow

Integration with pytest provides markers for harm categorization and statistical trials, automatic result collection, and terminal summaries. Tests can run across worker processes with pytest-xdist, and RAMPART produces a single unified report. Structured JSON output is intended for CI dashboards.