# llmware

> Open-source Python framework for building local, private RAG pipelines and agent workflows with small, specialized models.

llmware is an open-source Python framework from AI Bloks for building knowledge-based LLM applications that run locally, on edge devices or self-hosted. It combines a model catalog of 300+ models with an integrated RAG pipeline covering parsing, embedding, retrieval and prompting. It is distributed under the Apache-2.0 license and installs with pip.

## What It Is

llmware is a developer toolkit for retrieval augmented generation and multi-step agent workflows. Its README says it is optimized for AI PCs, laptops, edge and self-hosted deployment on Windows, Mac and Linux. It supports GGUF, OpenVINO, ONNXRuntime, ONNXRuntime-QNN, WindowsLocalFoundry and PyTorch behind a common interface. Cloud models from OpenAI, Anthropic and Google are also supported.

## How the RAG Pipeline Works

A Library is the knowledge-base container. Calling add_files routes mixed file types (pdf, pptx, docx, xlsx, txt, csv, md, json, wav, png, jpg, html) to parsers, which chunk the text and index it. Embeddings can be installed on a library with a chosen embedding model and vector database, and several can coexist. The Query class supports text, semantic, hybrid, metadata and custom-filter retrieval. Prompt with Sources packages retrieved passages or documents into model-ready context and offers post-inference evidence checks and prompt history.

## Models and Agents

The catalog includes the llmware-finetuned SLIM, BLING, DRAGON and Industry-BERT series. SLIM models are fine-tuned for function calling, and the LLMfx agent class loads them as tools such as sentiment, extract, topics and boolean. It aggregates the results into a report and activity summary. Many models ship in quantized GGUF versions for CPU use.

## Data Stores and Setup

Text collection databases are SQLite, MongoDB and Postgres. Supported vector databases are Milvus, PGVector, Neo4j, Redis, LanceDB, ChromaDB, Qdrant, FAISS, Pinecone and Mongo Atlas. A no-install option uses SQLite with ChromaDB. Docker Compose scripts are provided for the server-based options. OCR parsing needs optional Tesseract and Poppler packages.

## Features
- Model catalog with 300+ models
- RAG pipeline with document parsing, chunking and embedding
- Text, semantic and hybrid retrieval with filters
- Prompt with Sources and evidence checking
- SLIM function-calling models and LLMfx agents
- GGUF, OpenVINO, ONNXRuntime and PyTorch support
- Support for 3 text databases and 10 vector databases
- OCR for embedded document images
- Whisper CPP voice transcription
- Natural language to SQL with Slim-SQL

## Integrations
OpenAI, Anthropic, Google, Hugging Face, Milvus, PGVector, Postgres, MongoDB, SQLite, Neo4j, Redis, Qdrant, FAISS, LanceDB, ChromaDB, Pinecone, OpenVINO, ONNXRuntime, LM Studio, Ollama, Streamlit, Docker

## Platforms
WINDOWS, MACOS, LINUX, API, DEVELOPER_SDK, CLI

## Pricing
Open Source

## Version
v0.4.7

## Links
- Website: https://llmware-ai.github.io/llmware/
- Documentation: https://llmware-ai.github.io/llmware/
- Repository: https://github.com/llmware-ai/llmware
- EveryDev.ai: https://www.everydev.ai/tools/llmware