# spec-ptc

> Speculative programmatic tool calling (sPTC) library that queues and executes tool calls asynchronously while an LLM streams code, overlapping execution with generation time.

sPTC (Speculative Programmatic Tool Calling) is an open-source Python library by Alex Zhang that accelerates LLM agent harnesses by speculatively launching tool calls while the model is still generating code. It targets harness designs like Recursive Language Models (RLMs) and CodeAct that rely on programmatic tool-calling (PTC), where all tools are embedded as functions inside a single code REPL. The library is available on PyPI (`pip install spec-ptc`) and is licensed under MIT.

## What It Is

sPTC is a latency-reduction technique for LLM agent harnesses that use a code REPL as their primary tool interface. In a standard (baseline) setup, the LLM generates a full code block, then executes it serially — each tool call blocks until the previous one finishes. sPTC instead monitors the token stream as it arrives, detects completed function-call statements in the partially-generated code, and speculatively launches those calls as Futures. By the time the full code block is ready to execute, many expensive sub-LLM or sub-agent calls have already returned, so the `exec` step hits cached results almost immediately.

```
baseline   tokens──────────────────▶ exec: call₁──▶call₂──▶…──▶callₙ──▶ answer
spec-ptc   tokens──────────────────▶ exec: claim·claim·claim ──▶ answer
                 ╲ call₁ ▶▶▶ done ╱
                  ╲ call₂ ▶▶▶ done╱
```

## Core Architecture

The library centers on a `Speculator` object that tracks registered tools and manages a shadow REPL used for speculation:

- **`@spec.tool` decorator** — marks functions as speculatable (must be pure/side-effect-free) or non-speculatable (e.g., `send_report`).
- **`spec.turn()` context manager** — snapshots REPL locals into a discarded shadow fork for the duration of one turn.
- **`t.feed(delta)`** — accepts streamed tokens; closed statements trigger speculative launches immediately.
- **`spec.hooks()`** — injects claim-or-run wrappers under the same function names into the REPL namespace, so the actual `exec` call hits cached results or falls back transparently.

For out-of-process use, a daemon (`spec-ptc-daemon`) exposes the same shadow-plus-store over a Unix socket using four JSON-lines messages: `turn_begin`, `feed`, `resolve`, and `turn_end`. A minimal `SpecClient` (~60 lines, stdlib only) is provided for integration.

## Supported Harnesses and Plugins

The repository ships wrappers in a `plugins/` directory targeting:

- **RLM (Recursive Language Models)** — one-line patch: `from demo.rlm import patch_rlm; patch_rlm()`
- **Claude Code** (`PreToolUse` hook)
- **OpenCode**
- **Pi-mono**

The technique is designed to be harness-agnostic; any system that streams tokens into a code REPL and then executes the result can adopt sPTC with minimal changes.

## Update: v0.1.1 (async + hook fix)

The latest release, v0.1.1, was published on 2026-08-24 and addresses async support and a hook fix. The repository was created on 2026-08-21 and has accumulated 158 stars and 12 forks in its first week, signaling early community interest. Open issues stand at zero as of the last update on 2026-08-26.

## Why It Matters

Sub-LLM and sub-agent calls are typically the dominant cost in RLM-style harnesses — they block execution and inflate wall-clock latency. sPTC reframes generation time as useful compute time by overlapping speculative calls with token streaming. Because speculation is gated on the `pure=True` flag, side-effectful tools are never speculatively executed, preserving correctness. The approach requires no changes to the underlying model or REPL logic, only the insertion of claim-or-run hooks and a token feed loop.

## Features
- Speculative tool call execution during LLM token streaming
- Async and sync tool support via @spec.tool decorator
- Pure/side-effect-free tool gating for safe speculation
- Shadow REPL fork per turn for isolated speculation
- Out-of-process daemon with JSON-lines socket protocol
- SpecClient for harness integration (~60 lines, stdlib only)
- Plugins for Claude Code, OpenCode, Pi-mono, and RLM
- One-line RLM patch integration
- Claim-or-run hook injection into existing REPL namespaces
- PyPI installable (pip install spec-ptc)

## Integrations
Claude Code, OpenCode, Pi-mono, RLM (Recursive Language Models), CodeAct, Anthropic Claude (programmatic tool-calling), PyPI

## Platforms
CLI, API, DEVELOPER_SDK

## Pricing
Open Source

## Version
v0.1.1

## Links
- Website: https://github.com/alexzhang13/spec-ptc
- Repository: https://github.com/alexzhang13/spec-ptc
- EveryDev.ai: https://www.everydev.ai/tools/spec-ptc