DataFog Python

DataFog Python is an MIT-licensed library from DataFog for detecting and redacting personally identifiable information (PII) in text. It runs locally and includes ready-made enforcement points for AI agents and LLM gateways. The latest release listed on GitHub is v4.9.0.

What It Is

DataFog Python is an SDK and CLI that scans text for PII and returns findings or redacted output. The default regex engine covers structured entities such as email, phone, SSN, credit card, IP address, date and ZIP code. Optional spaCy and GLiNER engines add named-entity detection for unstructured text like person and organization names, and a smart engine cascades regex with the NER engines.

Agent and Gateway Firewall

The README describes two enforcement points that catch PII before it leaves the machine, without echoing matched values into logs or transcripts. The Claude Code hook (datafog-hook) gates agent tool calls such as shell commands, web requests, file writes and MCP tools, and warns the model when prompts or tool results contain PII. The LiteLLM guardrail (DataFogGuardrail) redacts or blocks PII in requests and responses for any LiteLLM-proxied provider. Both default to a high-precision entity set, and allowlists and allowlist regex patterns can exempt known-safe values.

Using It in Code

Helpers such as sanitize, scan_prompt, filter_output and create_guardrail cover scanning prompts, redacting model output and wrapping functions with a guardrail. German structured identifiers are opt-in through locale selection. The CLI offers scan, redact, replace (pseudonymize) and hash commands. Backward-compatible DataFog and TextService classes remain available.

Current Status and Direction

OCR and Spark support are deprecated in 4.9.0 and slated for removal in 5.0. The 4.9 line also adds opt-in Rust detection through DataFog Core and a native datafog.v5 preview. The DataFog site presents DataFog Core as a separate package and API with its own migration guide. Telemetry is disabled by default and does not include input text or detected values.