Detoxify

Detoxify is an open-source Python library from Unitary that provides trained models and code to predict toxic comments across three Jigsaw Kaggle challenges. It was built by Laura Hanu at Unitary and is released under the Apache 2.0 license. The latest listed release is v0.5.3.

What It Is

Detoxify is a toxic comment classification library. You pass in a string or a list of strings, and a model returns per-label toxicity scores. It is built on PyTorch Lightning and Hugging Face Transformers. The README states that its intended uses are research, fine-tuning on carefully constructed datasets, and helping content moderators flag harmful content faster.

Models and Labels

There are three main models, each tied to a Jigsaw challenge:

• original: bert-base-uncased, trained on the Toxic Comment Classification Challenge.
• unbiased: roberta-base, trained on the Unintended Bias in Toxicity Classification challenge.
• multilingual: xlm-roberta-base, trained on the Multilingual Toxic Comment Classification challenge.

Smaller original-small and unbiased-small variants built on Albert are also available. The multilingual model supports English, French, Spanish, Italian, Portuguese, Turkish and Russian. Labels include toxicity, severe toxicity, obscene, threat, insult, identity attack and sexual explicit, plus identity mention labels in the unbiased setup.

Workflow

Install with pip install detoxify, then call Detoxify('original').predict(...). A device such as CUDA can be specified, and the model defaults to CPU. A run_prediction.py script can score a single comment or a text file and save the results to CSV. Checkpoints can be loaded through torch.hub. The repository also includes training configs, a Kaggle data download workflow, and evaluation scripts, including a bias metric computation.

Limitations to Know

The README warns that comments containing swearing, insults or profanity are likely to be classified as toxic regardless of tone or intent, such as humour or self-deprecation. This could bias results against vulnerable minority groups. It links to resources on bias in hate speech detection.