FastEmbed

FastEmbed is a Python library maintained by Qdrant, originally created by Nirant Kasliwal, for generating embeddings. It runs models through ONNX Runtime rather than PyTorch and is released under the Apache 2.0 license. The latest listed release is v0.9.0.

What It Is

FastEmbed turns text and images into vectors for retrieval and search. Install it with pip, instantiate a model class such as TextEmbedding, and call embed() on a list of documents to get a generator of numpy vectors. The default text model is BAAI/bge-small-en-v1.5, which produces 384-dimensional vectors, and the project says it is based on Flag Embedding as shown on the MTEB leaderboard.

Supported Embedding and Reranking Types

• Dense text embeddings, including custom models added via add_custom_model.
• Sparse text embeddings such as SPLADE++.
• Late interaction text models such as ColBERT.
• Image embeddings, for example CLIP vision models.
• Late interaction multimodal models such as ColPali.
• Cross-encoder rerankers via TextCrossEncoder, which score query and document pairs and also accept custom models.

Light and Fast Design

The README states that FastEmbed needs no GPU and avoids downloading large PyTorch dependencies, making it a candidate for serverless runtimes such as AWS Lambda. It uses quantized model weights and data parallelism for encoding large datasets. The project claims it is faster than PyTorch-based approaches and more accurate than OpenAI Ada-002. A separate fastembed-gpu package enables GPU execution through CUDAExecutionProvider.

Use with Qdrant

Installing qdrant-client[fastembed] lets the Qdrant client embed documents and queries directly. You pass models.Document objects with a model name to create collections, upload points and run query_points. An in-memory mode is available for experimentation. The documentation also includes examples for hybrid search, binary quantization, multi-GPU use and a comparison with Hugging Face.