# Galahad

> An inference library that persists and restores KV cache for self-hosted models, reducing repeated prompt processing and supporting document retrieval and agent replay.

Galahad is Corbenic AI’s proprietary inference memory software for teams running their own language models. It stores previously computed KV cache and restores reusable state instead of repeating prompt processing. The software runs inside the customer’s infrastructure and is distributed through the galahad-kv Python package.

## What It Is

Galahad provides a C++ core, an in-process C API, and connectors for vLLM and SGLang. Its documented components combine exact reuse of prior model work with persistent storage; the repository also describes document retrieval through Blaise and llama.cpp integration. These mechanisms are intended to reduce repeated computation rather than train or replace the underlying model.

## Developer Workflows

The documented agent tooling includes step recording, replay comparisons through pytest, failure attribution, copy-on-write branches, and an inspector with rewind. Connectors are described for LangGraph, LangChain, CrewAI, and custom loops. Developers can use these facilities to investigate changes in agent behavior and reuse common starting context.

## Deployment and Isolation

The current package is a beta for Linux x86-64 with Python 3.10–3.14 and an NVIDIA GPU. Installation guidance pins vLLM 0.30 and SGLang 0.5.20. SGLang’s hierarchical cache also needs sufficient host RAM. Importantly, the package treats one serving process as one trust domain: mutually untrusted customers need separate processes, stores, and keys. Saved cache is encrypted, and operators must preserve their local key material.

## Performance Evidence

Corbenic publishes workload-specific measurements for cache restoration, repeated queries, and long documents. These are vendor results rather than universal guarantees; the company offers a measured pilot on the customer’s own servers. The public GitHub repository and downloadable wheel do not establish an open-source licence.

## Features
- Persistent KV cache across requests and restarts
- Byte-exact cache restoration described by the vendor
- vLLM connector and SGLang HiCache backend
- C++ library with C API
- Document storage and retrieval
- Agent recording and pytest replay comparisons
- Failure attribution and copy-on-write branches
- Inspector and optional rewind
- Prefix sharing and block pinning
- Encryption at rest and malformed-cache checks
- GPU-time and cost-savings reporting

## Integrations
vLLM, SGLang, llama.cpp, LangGraph, LangChain, CrewAI, pytest

## Platforms
LINUX, API, DEVELOPER_SDK, CLI

## Pricing
Freemium — Free tier available with paid upgrades

## Version
1.31.5

## Links
- Website: https://www.corbenic.ai/en/galahad
- Documentation: https://github.com/corbenicai/galahad/wiki
- Repository: https://github.com/corbenicai/galahad
- EveryDev.ai: https://www.everydev.ai/tools/galahad