# Giskard Open Source

> Open-source Python library for evaluating, red teaming, and testing LLM agents and RAG systems.

Giskard Open Source is a Python library from Giskard AI for testing and evaluating agentic systems. Version 3 is a rewrite built around modular packages, with a checks library for evals and a scan package for vulnerability and RAG quality scanning. It is released under the Apache 2.0 license and requires Python 3.12 or newer.

## What It Is

Giskard wraps any target, whether an LLM, a black-box agent, or a multi-step pipeline, and tests it with scenarios and checks. Evals are designed for non-deterministic outputs, where the same input can produce different valid responses. The library sits alongside the commercial Giskard Hub, which the docs describe as an enterprise platform with a UI and SDK.

## Giskard Checks

`giskard-checks` provides a scenario API in which a target is called through interactions and the traces are validated by checks. Built-in checks include string matching, comparisons, regex, semantic similarity, and LLM-as-judge checks such as Groundedness, Conformity, and LLMJudge. Scenarios can be grouped into suites and can cover multi-turn conversations. The README lists use cases such as catching regressions, validating RAG grounding, and enforcing safety rules.

## Giskard Scan

`giskard-scan` is installed with the `scan` extra. `vulnerability_scan` generates adversarial test suites from a plain-language description of the agent, covering prompt injection, harmful content, stereotypes, and misinformation, and it supports custom scenario generators. `quality_scan` evaluates knowledge-base quality for RAG systems. The docs note that scan results are not a safety or compliance guarantee, and that LLM judges and generators need a provider extra and an API key.

## Version Notes

Giskard v2 remains available but is no longer actively maintained. The legacy automatic scan for tabular and ML models exists only in v2. Telemetry is optional and aggregated, it sends no prompts or outputs, and it can be disabled with an environment variable.

## Features
- Scenario and check based evals for LLM agents
- LLM-as-judge checks (Groundedness, Conformity, LLMJudge)
- Multi-turn scenario testing
- Test suites
- Automated vulnerability scanning and red teaming (vulnerability_scan)
- RAG quality evaluation (quality_scan)
- Custom scenario generators
- Prompt injection probes
- Optional telemetry with opt-out

## Integrations
OpenAI, Anthropic, pytest

## Platforms
API, DEVELOPER_SDK, CLI

## Pricing
Open Source

## Version
giskard-checks v1.0.6

## Links
- Website: https://docs.giskard.ai
- Documentation: https://docs.giskard.ai/oss
- Repository: https://github.com/Giskard-AI/giskard-oss
- EveryDev.ai: https://www.everydev.ai/tools/giskard-oss