# Heretic

> Fully automatic censorship removal for transformer-based language models using directional ablation and TPE-based parameter optimization.

Heretic is a free and open-source command-line tool created by Philipp Emanuel Weidmann that removes safety alignment (censorship) from transformer-based language models without requiring expensive post-training. It is released under the GNU Affero General Public License v3.0 and is installable via pip as `heretic-llm`. The project reached v1.4.0 as of June 2026 and has accumulated over 30,000 GitHub stars.

## What It Is

Heretic implements a parametrized variant of directional ablation — also known as "abliteration" — to suppress refusal behavior in language models. Rather than requiring manual tuning or an understanding of transformer internals, Heretic uses a TPE-based parameter optimizer powered by Optuna to automatically find high-quality ablation parameters. The optimization co-minimizes the number of refusals and the KL divergence from the original model, aiming to produce a decensored model that retains as much of the original model's intelligence as possible. Anyone who can run a command-line program can use it.

## How the Ablation Process Works

Heretic identifies attention out-projection and MLP down-projection matrices in each transformer layer and orthogonalizes them with respect to a computed "residual direction," inhibiting the expression of that direction in the result. Key optimizable parameters include:

- **`direction_index`**: Index of a residual direction, or `per layer` to use each layer's own direction
- **`max_weight`, `max_weight_position`, `min_weight`, `min_weight_distance`**: Parameters describing the shape and position of the ablation weight kernel across layers

The residual direction for each layer is computed as a difference-of-means between first-token residuals for "harmful" and "harmless" example prompts. Heretic's main innovations include a highly flexible ablation weight kernel shape, a float-valued direction index enabling linear interpolation between direction vectors, and separate ablation parameters for each component type.

## Model Support and Hardware

Heretic supports most dense transformer models, including many multimodal models, several MoE architectures, and hybrid models like Qwen3.5. Pure state-space models and certain research architectures are not yet supported out of the box. The tool supports model quantization via bitsandbytes (4-bit), which can significantly reduce VRAM requirements. On an RTX 3090 with default configuration, decensoring a 4B-parameter model takes approximately 20–30 minutes. PyTorch 2.2+ and Python 3.10+ are required.

## Research Features

With the optional `research` extra (`pip install 'heretic-llm[research]'`), Heretic provides interpretability tooling:

- **`--plot-residuals`**: Computes residual vectors per layer, performs PaCMAP projection to 2D, and generates scatter plots and animated GIFs showing how residuals transform between layers
- **`--print-residual-geometry`**: Outputs a detailed table of cosine similarities, L2 norms, and silhouette coefficients for harmful/harmless residual clusters per layer

## Update: v1.4.0

The latest release is v1.4.0, published June 14, 2026. The repository was created in September 2025 and has seen active development, with the last push recorded in September 2026. The project publishes decensored model weights to Hugging Face under the `heretic-org` organization. According to the project's GitHub README, the community has created and published well over 5,000 models using Heretic on Hugging Face. The project was ranked #1 Repository of the Day on Trendshift.

## Features
- Fully automatic censorship removal (abliteration) for LLMs
- TPE-based parameter optimizer powered by Optuna
- Directional ablation with flexible weight kernel shape
- Float-valued direction index with linear interpolation
- Separate ablation parameters per transformer component
- Support for dense, multimodal, MoE, and hybrid models
- 4-bit quantization support via bitsandbytes
- Built-in evaluation functionality (refusal rate, KL divergence)
- Hugging Face model upload integration
- Interactive chat mode for testing decensored models
- Standard benchmark support (MMLU, GSM8K)
- Residual vector plotting with PaCMAP projection (research extra)
- Residual geometry analysis table (research extra)
- TOML-based configuration file support
- Automatic hardware benchmarking for optimal batch size

## Integrations
Hugging Face Hub, Optuna, PyTorch, bitsandbytes, PaCMAP, uv (dependency management)

## Platforms
WINDOWS, MACOS, LINUX, WEB, API, CLI

## Pricing
Open Source

## Version
v1.4.0

## Links
- Website: https://heretic-project.org
- Documentation: https://heretic-project.org/tutorial
- Repository: https://github.com/p-e-w/heretic
- EveryDev.ai: https://www.everydev.ai/tools/heretic