# DataFog Python

> Python library for offline PII detection and redaction, with a Claude Code hook and LiteLLM guardrail for AI agents and LLM apps.

DataFog Python is an MIT-licensed library from DataFog for detecting and redacting personally identifiable information (PII) in text. It runs locally and includes ready-made enforcement points for AI agents and LLM gateways. The latest release listed on GitHub is v4.9.0.

## What It Is

DataFog Python is an SDK and CLI that scans text for PII and returns findings or redacted output. The default regex engine covers structured entities such as email, phone, SSN, credit card, IP address, date and ZIP code. Optional spaCy and GLiNER engines add named-entity detection for unstructured text like person and organization names, and a smart engine cascades regex with the NER engines.

## Agent and Gateway Firewall

The README describes two enforcement points that catch PII before it leaves the machine, without echoing matched values into logs or transcripts. The Claude Code hook (`datafog-hook`) gates agent tool calls such as shell commands, web requests, file writes and MCP tools, and warns the model when prompts or tool results contain PII. The LiteLLM guardrail (`DataFogGuardrail`) redacts or blocks PII in requests and responses for any LiteLLM-proxied provider. Both default to a high-precision entity set, and allowlists and allowlist regex patterns can exempt known-safe values.

## Using It in Code

Helpers such as `sanitize`, `scan_prompt`, `filter_output` and `create_guardrail` cover scanning prompts, redacting model output and wrapping functions with a guardrail. German structured identifiers are opt-in through locale selection. The CLI offers scan, redact, replace (pseudonymize) and hash commands. Backward-compatible `DataFog` and `TextService` classes remain available.

## Current Status and Direction

OCR and Spark support are deprecated in 4.9.0 and slated for removal in 5.0. The 4.9 line also adds opt-in Rust detection through DataFog Core and a native `datafog.v5` preview. The DataFog site presents DataFog Core as a separate package and API with its own migration guide. Telemetry is disabled by default and does not include input text or detected values.

## Features
- Regex-based structured PII detection
- Optional spaCy and GLiNER NER engines
- Smart engine cascading regex with NER
- Claude Code hook for gating agent tool calls
- LiteLLM gateway guardrail
- Redaction, pseudonymization and hashing
- Allowlists and allowlist regex patterns
- Opt-in German structured PII via locales
- CLI for scan, redact, replace and hash
- Guardrail decorator for LLM functions
- Optional OCR and Spark support (deprecated in 4.9)
- Opt-in telemetry, off by default

## Integrations
Claude Code, LiteLLM, spaCy, GLiNER, Presidio (entity name aliases), PySpark

## Platforms
WINDOWS, MACOS, LINUX, WEB, API, DEVELOPER_SDK, CLI

## Pricing
Open Source

## Version
4.9.0

## Links
- Website: https://datafog.ai
- Documentation: https://docs.datafog.ai/
- Repository: https://github.com/DataFog/datafog-python
- EveryDev.ai: https://www.everydev.ai/tools/datafog-python