coding.sh
Open-source, local-first observability and evals tool that records what AI coding agents did and what it cost.
At a Glance
Free, MIT-licensed, local-first observability for AI agents; runs on your machine with no account.
Engagement
Available On
Listed Nov 2026
About coding.sh
coding.sh is an open-source, local-first observability layer for AI agents, published by jackulau on GitHub under the MIT license. It records each prompt, model call, tool call, file change and cost from coding agents such as Claude Code, Codex and Cursor, and stores sessions on the user's machine. The project is pre-1.0, and the docs note that commands and file formats can still change between releases.
What It Is
coding.sh captures agent runs through hooks, transcripts, plugins, a wrapper command or OpenTelemetry, redacts secrets at ingest, and keeps each session in a local SQLite index under ~/.coding.sh. Users read sessions in a local viewer (coding.sh ui), query them from the CLI, or let their agents read their own history over MCP. Own apps can send traces through a TypeScript SDK, a Python SDK or OpenTelemetry.
From Recording to Evals
Failed sessions can be turned into replayable evals with coding.sh eval new. Experiments run over versioned datasets using code graders or calibrated LLM judges, and coding.sh exp compare A/B tests prompts, harness files such as AGENTS.md, and models with paired statistics. A benchmark command draws new cases daily, runs every contender on the same frozen set and writes a static report.
Setup Path
Running npx coding.sh or the curl install script finds installed agents, shows a plan, asks once, backs up config files it edits, and imports past sessions. Starting a new agent session then records it. coding.sh uninstall removes the hooks and MCP entries.
Privacy and Production Use
With no sinks configured, no session data leaves the machine; the only default request is a daily update check that can be disabled. For teams, coding.sh serve runs ingest, the viewer and the API on one port, backed by SQLite, Postgres or ClickHouse. A gateway command links calls made through LLM gateways such as OpenRouter, LiteLLM and Portkey to their billed cost.
Current Status
The README lists the latest release as 0.1.7. Cursor CLI and Amp adapters are documented as not yet run on a live install, and capture from several agents has not yet been checked on Windows.
Community Discussions
Be the first to start a conversation about coding.sh
Share your experience with coding.sh, ask questions, or help others learn from your insights.
Pricing
Open Source
Free, MIT-licensed, local-first observability for AI agents; runs on your machine with no account.
- MIT licensed
- Runs on your machine, no account
- Session, cost and token tracking
- Local viewer (coding.sh ui)
- Evals and experiments
Capabilities
Key Features
- Session recording of prompts, model calls, tool calls and file changes
- Per-session, agent, model and day cost and token tracking
- Local web viewer with step-by-step timeline
- Turn failed sessions into replayable evals
- Experiments with paired statistics and LLM judges
- Daily benchmarks with static reports
- MCP server so agents can read their own runs
- TypeScript and Python SDKs
- OpenTelemetry collector
- Secret redaction at ingest
- Self-hosted team server with SQLite, Postgres or ClickHouse
- LLM gateway cost linking
