INSTRUCTOR

INSTRUCTOR is an instruction-finetuned text embedding model from the xlang-ai organization, released with the ACL 2023 paper "One Embedder, Any Task: Instruction-Finetuned Text Embeddings". It generates embeddings tailored to a task and domain by pairing each text with a short instruction. The repository provides the Python package, training code, evaluation scripts, and pretrained checkpoints.

What It Is

INSTRUCTOR is a text embedding toolkit distributed as the InstructorEmbedding Python package. Users load a pretrained checkpoint and call encode on instruction-text pairs, such as "Represent the Science title:" followed by the text. The output is a list of numpy arrays or PyTorch tensors. The authors state it achieves state-of-the-art results on 70 diverse embedding tasks.

How Instructions Shape Embeddings

Instructions follow a template: Represent the domain text_type for task_objective. The text type is required, while domain and task objective are optional. The same model can therefore serve classification, retrieval, clustering, and text evaluation. The README includes examples for computing similarities between texts, retrieving documents for a query, and clustering sentences.

Models, Training and Evaluation

Three checkpoints are listed: instructor-base, instructor-large, and instructor-xl. The training data, MEDI, combines 330 datasets, and a training script is provided. Evaluation scripts cover the MTEB, Billboard, and Prompt Retrieval benchmarks. The README also shows dynamic quantization with PyTorch, which it says reduces model size by about 10x.

Ecosystem

The README notes community integrations including LangChain, MosaicML, embaas, and Haystack. The code is released under the Apache 2.0 license.