# ShardMemo

> Research implementation of a three-tier sharded memory service for LLM agents that applies scope predicates before routing retrieval.

ShardMemo is the official implementation of the paper "ShardMemo: Scope-Before-Routing for Agentic Memory Retrieval" (listed as Findings of EMNLP 2026). It is a Python memory service for LLM agents that stores evidence in shards and restricts which shards a request may access before any routing happens. The repository includes experiment scripts for LoCoMo, HotpotQA and ToolBench.

## What It Is

ShardMemo is a three-tier agentic memory architecture. Tier A keeps a bounded active context. Tier B stores evidence in shards, each with a shard-local approximate nearest neighbor index. Tier C holds a versioned skill library with skill induction, retrieval and validated execution.

## How Retrieval Works

In Tier B, metadata predicates first determine which shards a request is allowed to access. A learned masked router then probes a few of the admissible shards, so inadmissible shards never consume probe budget. A controller gates between tiers and manages the read path across all three. Ranking fuses BM25 with embedding-based evidence scores.

## Running the Experiments

Setup requires Python 3.11 or newer, installing the pinned requirements and copying `.env.example` to `.env`. Answers are generated by any LLM behind an OpenAI-compatible API, configured through `LLM_API_KEY`, `LLM_BASE_URL` and `LLM_MODEL`. Scripts `run_locomo.py`, `run_hotpotqa.py` and `run_toolbench.py` cover ablations, probe-budget sweeps, shard-count scaling, label-noise robustness, long-context HotpotQA at several sizes, and Tier C skill retrieval. Each run writes a `summary.json` and a `report.md`, with embeddings and LLM responses cached locally. Settings live in `shardmemo/config.py`.

## Features
- Three-tier memory: active context, sharded evidence, versioned skill library
- Metadata-predicate scoping before routing
- Learned masked router with Top-B probe and adaptive Top-P selection
- Shard-local HNSW/flat approximate nearest neighbor indexes
- BM25 and fused evidence ranking
- Skill induction, versioning and validated execution
- Experiment scripts for LoCoMo, HotpotQA and ToolBench
- Cached embeddings and LLM responses
- OpenAI-compatible LLM backend

## Integrations
OpenAI-compatible APIs, LoCoMo, HotpotQA, ToolBench, PyTorch

## Platforms
API, DEVELOPER_SDK, CLI

## Pricing
Open Source

## Links
- Website: https://github.com/Erikdai/shardmemo
- Documentation: https://github.com/Erikdai/shardmemo
- Repository: https://github.com/Erikdai/shardmemo
- EveryDev.ai: https://www.everydev.ai/tools/shardmemo