Model2Vec-rs

Model2Vec-rs is the official Rust implementation of Model2Vec inference, published by MinishLab (Minish) as an MIT-licensed crate. It generates text embeddings from Model2Vec static embedding models, either inside Rust applications or through a command-line tool. The latest release listed is v0.3.0.

What It Is

Model2Vec-rs is an inference engine for Model2Vec static embedding models, which are compact models distilled from sentence transformers. It handles inference only. Distillation, training and fine-tuning stay in the Python model2vec package. Developers load a model by Hugging Face Hub ID or from a local path, then encode sentences into embeddings returned as vectors of f32 values.

How You Use It

As a library, you add the crate with cargo, load a model with StaticModel::from_pretrained, and call encode or encode_with_args. The second call sets a maximum token length and batch size. Defaults are a max length of 512 and a batch size of 1024. As a CLI, you install with the cli feature. encode-single prints one embedding as JSON, and encode reads a text file and writes embeddings to a JSON output file.

Deployment Options and Performance

Models can be loaded from raw bytes or owned/borrowed data, which suits embedded and WebAssembly use. Feature flags include local-only to disable remote downloads, fancy-regex as an alternative tokenizer backend, and wasm for in-memory loading. Supported weight types are f32, f16 and i8 in safetensors files. The README reports a single-threaded CPU benchmark of 8000 samples per second for Rust against 4650 for Python, roughly 1.7x faster. Its model table lists English, multilingual and code-focused potion models.