spec-ptc

sPTC (Speculative Programmatic Tool Calling) is an open-source Python library by Alex Zhang that accelerates LLM agent harnesses by speculatively launching tool calls while the model is still generating code. It targets harness designs like Recursive Language Models (RLMs) and CodeAct that rely on programmatic tool-calling (PTC), where all tools are embedded as functions inside a single code REPL. The library is available on PyPI (pip install spec-ptc) and is licensed under MIT.

What It Is

sPTC is a latency-reduction technique for LLM agent harnesses that use a code REPL as their primary tool interface. In a standard (baseline) setup, the LLM generates a full code block, then executes it serially — each tool call blocks until the previous one finishes. sPTC instead monitors the token stream as it arrives, detects completed function-call statements in the partially-generated code, and speculatively launches those calls as Futures. By the time the full code block is ready to execute, many expensive sub-LLM or sub-agent calls have already returned, so the exec step hits cached results almost immediately.

baseline   tokens──────────────────▶ exec: call₁──▶call₂──▶…──▶callₙ──▶ answer
spec-ptc   tokens──────────────────▶ exec: claim·claim·claim ──▶ answer
                 ╲ call₁ ▶▶▶ done ╱
                  ╲ call₂ ▶▶▶ done╱

Core Architecture

The library centers on a Speculator object that tracks registered tools and manages a shadow REPL used for speculation:

• @spec.tool decorator — marks functions as speculatable (must be pure/side-effect-free) or non-speculatable (e.g., send_report).
• spec.turn() context manager — snapshots REPL locals into a discarded shadow fork for the duration of one turn.
• t.feed(delta) — accepts streamed tokens; closed statements trigger speculative launches immediately.
• spec.hooks() — injects claim-or-run wrappers under the same function names into the REPL namespace, so the actual exec call hits cached results or falls back transparently.

For out-of-process use, a daemon (spec-ptc-daemon) exposes the same shadow-plus-store over a Unix socket using four JSON-lines messages: turn_begin, feed, resolve, and turn_end. A minimal SpecClient (~60 lines, stdlib only) is provided for integration.

Supported Harnesses and Plugins

The repository ships wrappers in a plugins/ directory targeting:

• RLM (Recursive Language Models) — one-line patch: from demo.rlm import patch_rlm; patch_rlm()
• Claude Code (PreToolUse hook)
• OpenCode
• Pi-mono

The technique is designed to be harness-agnostic; any system that streams tokens into a code REPL and then executes the result can adopt sPTC with minimal changes.

Update: v0.1.1 (async + hook fix)

The latest release, v0.1.1, was published on 2026-08-24 and addresses async support and a hook fix. The repository was created on 2026-08-21 and has accumulated 158 stars and 12 forks in its first week, signaling early community interest. Open issues stand at zero as of the last update on 2026-08-26.

Why It Matters

Sub-LLM and sub-agent calls are typically the dominant cost in RLM-style harnesses — they block execution and inflate wall-clock latency. sPTC reframes generation time as useful compute time by overlapping speculative calls with token streaming. Because speculation is gated on the pure=True flag, side-effectful tools are never speculatively executed, preserving correctness. The approach requires no changes to the underlying model or REPL logic, only the insertion of claim-or-run hooks and a token feed loop.