WhisperX
Open-source speech recognition tool built on Whisper that adds word-level timestamps, VAD-based batching and speaker diarization.
At a Glance
WhisperX core is released under the OSI-approved BSD 2-Clause license and can be self-hosted at no cost.
Engagement
Available On
Listed Nov 2026
About WhisperX
WhisperX is an open-source automatic speech recognition project by Max Bain, developed with support from the Visual Geometry Group at the University of Oxford. It transcribes audio with word-level timestamps and optional speaker diarization, and is installed from PyPI as a command-line tool and Python library. The latest listed release is v3.8.6, published 2026-05-25.
What It Is
WhisperX extends OpenAI's Whisper ASR model. Whisper produces utterance-level timestamps that can be off by several seconds and does not natively support batching. WhisperX addresses this with voice-activity-detection preprocessing, batched inference through a faster-whisper backend, and forced phoneme alignment using wav2vec2-style models. The README states this yields 70x realtime transcription with large-v2 and needs under 8GB of GPU memory with beam_size=5.
How the Pipeline Works
The workflow has three stages. First, audio is transcribed in batches with Whisper. Second, the segments are aligned with a language-specific phoneme model to give per-word timings. Third, speaker labels from pyannote-audio diarization are assigned to words and segments. Output includes subtitle files, and --highlight_words visualises word timings in the .srt file. An interleaved-context option carries context between batches to improve punctuation and technical terms on long audio.
Setup and Usage
Install with pip install whisperx or run it via uvx. GPU use calls for CUDA toolkit 12.8, while CPU runs (including on macOS) use --device cpu --compute_type int8. Diarization needs a Hugging Face token and acceptance of the pyannote speaker-diarization-community-1 model agreement. Installing Numba (whisperx[fast]) speeds up alignment on long files. To reduce GPU memory, lower the batch size, use a smaller model, or use int8 compute.
Language Support and Limitations
Default alignment models cover English, French, German, Spanish and Italian through torchaudio, with many more languages through Hugging Face. Other languages need a phoneme-based model supplied by the user. Documented limitations: words without characters in the alignment dictionary (such as "2014." or "£13.60") get no timing, overlapping speech is handled poorly, diarization is imperfect, and a language-specific wav2vec2 model is required.
Community Discussions
Be the first to start a conversation about WhisperX
Share your experience with WhisperX, ask questions, or help others learn from your insights.
Pricing
Open Source
WhisperX core is released under the OSI-approved BSD 2-Clause license and can be self-hosted at no cost.
- BSD 2-Clause "Simplified" License (BSD-2-Clause)
- Batched inference for 70x realtime transcription using whisper large-v2
- Word-level timestamps using wav2vec2 alignment
- Speaker diarization via pyannote-audio
- VAD preprocessing
Capabilities
Key Features
- Batched Whisper inference with faster-whisper backend
- Word-level timestamps via wav2vec2 forced alignment
- Speaker diarization using pyannote-audio
- VAD preprocessing to reduce hallucination
- Interleaved context batching for long audio
- Multilingual alignment model selection
- CPU and GPU execution with configurable compute types
- Optional Numba-accelerated alignment
- Command-line and Python APIs
