ShardMemo

ShardMemo is the official implementation of the paper "ShardMemo: Scope-Before-Routing for Agentic Memory Retrieval" (listed as Findings of EMNLP 2026). It is a Python memory service for LLM agents that stores evidence in shards and restricts which shards a request may access before any routing happens. The repository includes experiment scripts for LoCoMo, HotpotQA and ToolBench.

What It Is

ShardMemo is a three-tier agentic memory architecture. Tier A keeps a bounded active context. Tier B stores evidence in shards, each with a shard-local approximate nearest neighbor index. Tier C holds a versioned skill library with skill induction, retrieval and validated execution.

How Retrieval Works

In Tier B, metadata predicates first determine which shards a request is allowed to access. A learned masked router then probes a few of the admissible shards, so inadmissible shards never consume probe budget. A controller gates between tiers and manages the read path across all three. Ranking fuses BM25 with embedding-based evidence scores.

Running the Experiments

Setup requires Python 3.11 or newer, installing the pinned requirements and copying .env.example to .env. Answers are generated by any LLM behind an OpenAI-compatible API, configured through LLM_API_KEY, LLM_BASE_URL and LLM_MODEL. Scripts run_locomo.py, run_hotpotqa.py and run_toolbench.py cover ablations, probe-budget sweeps, shard-count scaling, label-noise robustness, long-context HotpotQA at several sizes, and Tier C skill retrieval. Each run writes a summary.json and a report.md, with embeddings and LLM responses cached locally. Settings live in shardmemo/config.py.