PyRIT

PyRIT (Python Risk Identification Tool) is an open source framework from Microsoft for security professionals and engineers who want to proactively identify risks in generative AI systems. It supports automated and human-led red teaming through a command-line scanner, a web GUI called CoPyRIT, and a modular Python framework. The latest stable release listed is v1.1.0.

What It Is

PyRIT is a red teaming toolkit for assessing the security and safety of generative AI systems at scale. Users configure a target model or endpoint, run attacks or scenarios against it, and score the responses. Results are stored in a built-in memory layer that tracks conversations, scores, and attack results.

Three Ways to Use It

• Scanner: run built-in scenarios from the command line with pyrit_scan or the interactive pyrit_shell, for example pyrit_scan run airt.scam --target openai_chat.
• CoPyRIT GUI: a local web app started with pyrit_backend for human-led red teaming, where users chat with AI systems, track findings, and collaborate with their team.
• Framework: modular components such as targets, converters, scorers, and memory, for building custom attacks in Python.

Attacks, Targets, and Scoring

The documentation describes multi-turn strategies such as Crescendo, TAP, and Skeleton Key, plus single-turn attacks. A scenario framework runs standardized evaluations covering content harms, psychosocial risks, data leakage, and more. Targets include OpenAI, Azure, Anthropic, Google, HuggingFace, custom HTTP or WebSocket endpoints, and web apps via Playwright. Scorers include true/false, Likert scale, classification, and custom scorers, powered by LLMs, Azure AI Content Safety, or custom logic. Memory can use SQLite or Azure SQL.

Setup Path

Install with pip install pyrit; Python 3.13 is recommended for local installation. Endpoint credentials go in ~/.pyrit/.env, and startup configuration goes in ~/.pyrit/.pyrit_conf.