FlashRank

FlashRank is an open-source Python library created by Prithivi Da that re-ranks search results before they are passed to LLMs. It runs on CPU without Torch or Transformers and supports cross-encoder (pairwise) and LLM-based (listwise) rerankers. The latest listed release is 0.2.9 (published 2024-11-29).

What It Is

FlashRank is a re-ranking layer for existing search and retrieval pipelines. Given a query and a list of candidate passages (each with an id, text and optional metadata), it returns the passages reordered by relevance score. It can sit behind lexical search, vector search or RAG pipelines, and hybrid search setups.

Models and Ranking Modes

Pairwise/pointwise rerankers are cross-encoder based with a 512-token maximum, while listwise rerankers are LLM based with an 8192-token maximum. Supported models listed in the README include ms-marco-TinyBERT-L-2-v2 (default, about 4MB), ms-marco-MiniLM-L-12-v2, rank-T5-flan, ms-marco-MultiBERT-L-12 (100+ languages), ce-esci-MiniLM-L12-v2, rank_zephyr_7b_v1_full (4-bit quantised GGUF) and miniReranker_arabic_v1. The README notes the current rank_zephyr integration supports a maximum of 20 passages in one pass.

Setup and Tuning

Install with pip install flashrank for lightweight rerankers, or pip install flashrank[listwise] for LLM-based ones. You create a Ranker with a model name, an optional cache_dir and a max_length, then call rerank with a RerankRequest. The README advises setting max_length just above the longest query-plus-passage token count, since larger values slow response time.

Serverless Deployment

In read-only serverless environments such as AWS Lambda, the README suggests creating a custom directory in the Dockerfile and pointing cache_dir at it for loading models and caching between warm calls. The project says its small package size shortens cold starts and redeployments.