EveryDev.ai
Subscribe
Home
Tools

5,901+ AI tools

  • Featured
  • Gems
  • Trending
  • New
  • Rate tools
  • Compare
  • Arena
Categories
  • Agents2972
  • Coding2248
  • Infrastructure1067
  • Projects818
  • MCP794
  • Research744
  • Analytics565
  • Marketing527
  • Design488
  • Testing476
  • Data442
  • Security412
  • Integration288
  • Communication284
  • Prompts283
  • Extensions238
  • Voice213
  • DevOps190
  • Commerce183
  • Web128
  • Learning123
  • Finance44
AI Tools by Topic
  • AI Coding Assistants
  • Agent Frameworks
  • MCP Servers
  • AI Prompt Tools
  • Vibe Coding Tools
  • AI Design Tools
  • AI Database Tools
  • AI Website Builders
  • AI Testing Tools
  • LLM Evaluations
Follow Us
  • X / Twitter
  • LinkedIn
  • Reddit
  • Discord
  • Threads
  • Bluesky
  • Mastodon
  • YouTube
  • GitHub
  • Instagram
Get Started
  • Users
  • Rate Tools
  • About
  • Editorial Standards
  • Corrections & Disclosures
  • Community Guidelines
  • Advertise
  • Contact Us
  • Newsletter
  • Submit a Tool
  • Start a Discussion
  • Write A Blog
  • Share A Build
  • Terms of Service
  • Privacy Policy
Explore with AI
  • ChatGPT
  • Gemini
  • Claude
  • Grok
  • Perplexity
Agent Experience
  • llms.txt
Theme
With AI, Everyone is a Dev. EveryDev.ai ©
    1. Home
    2. Tools
    3. coding.sh
    coding.sh icon

    coding.sh

    Monitoring Tools
    Featured

    Open-source, local-first observability and evals tool that records what AI coding agents did and what it cost.

    Visit Website

    At a Glance

    Pricing
    Open Source

    Free, MIT-licensed, local-first observability for AI agents; runs on your machine with no account.

    Engagement

    Available On

    Windows
    macOS
    Linux
    Web
    API

    Resources

    WebsiteDocsGitHubllms.txt

    Topics

    Monitoring ToolsObservability PlatformsLLM Evaluations

    Alternatives

    EvidentlyLangtraceTraceloop
    Developer
    Jack LauAmes, IAEst. 2021

    Listed Nov 2026

    About coding.sh

    coding.sh is an open-source, local-first observability layer for AI agents, published by jackulau on GitHub under the MIT license. It records each prompt, model call, tool call, file change and cost from coding agents such as Claude Code, Codex and Cursor, and stores sessions on the user's machine. The project is pre-1.0, and the docs note that commands and file formats can still change between releases.

    What It Is

    coding.sh captures agent runs through hooks, transcripts, plugins, a wrapper command or OpenTelemetry, redacts secrets at ingest, and keeps each session in a local SQLite index under ~/.coding.sh. Users read sessions in a local viewer (coding.sh ui), query them from the CLI, or let their agents read their own history over MCP. Own apps can send traces through a TypeScript SDK, a Python SDK or OpenTelemetry.

    From Recording to Evals

    Failed sessions can be turned into replayable evals with coding.sh eval new. Experiments run over versioned datasets using code graders or calibrated LLM judges, and coding.sh exp compare A/B tests prompts, harness files such as AGENTS.md, and models with paired statistics. A benchmark command draws new cases daily, runs every contender on the same frozen set and writes a static report.

    Setup Path

    Running npx coding.sh or the curl install script finds installed agents, shows a plan, asks once, backs up config files it edits, and imports past sessions. Starting a new agent session then records it. coding.sh uninstall removes the hooks and MCP entries.

    Privacy and Production Use

    With no sinks configured, no session data leaves the machine; the only default request is a daily update check that can be disabled. For teams, coding.sh serve runs ingest, the viewer and the API on one port, backed by SQLite, Postgres or ClickHouse. A gateway command links calls made through LLM gateways such as OpenRouter, LiteLLM and Portkey to their billed cost.

    Current Status

    The README lists the latest release as 0.1.7. Cursor CLI and Amp adapters are documented as not yet run on a live install, and capture from several agents has not yet been checked on Windows.

    coding.sh - 1

    Community Discussions

    Be the first to start a conversation about coding.sh

    Share your experience with coding.sh, ask questions, or help others learn from your insights.

    Pricing

    OPEN SOURCE

    Open Source

    Free, MIT-licensed, local-first observability for AI agents; runs on your machine with no account.

    • MIT licensed
    • Runs on your machine, no account
    • Session, cost and token tracking
    • Local viewer (coding.sh ui)
    • Evals and experiments

    Capabilities

    Key Features

    • Session recording of prompts, model calls, tool calls and file changes
    • Per-session, agent, model and day cost and token tracking
    • Local web viewer with step-by-step timeline
    • Turn failed sessions into replayable evals
    • Experiments with paired statistics and LLM judges
    • Daily benchmarks with static reports
    • MCP server so agents can read their own runs
    • TypeScript and Python SDKs
    • OpenTelemetry collector
    • Secret redaction at ingest
    • Self-hosted team server with SQLite, Postgres or ClickHouse
    • LLM gateway cost linking

    Integrations

    Claude Code
    Codex
    Cursor
    OpenCode
    Antigravity CLI
    GitHub Copilot CLI
    Amp
    Goose
    Pi
    Factory Droid
    Aider
    T3 Code
    OpenTelemetry
    MCP
    OpenAI Agents SDK
    LangGraph
    Vercel AI SDK
    Claude Agent SDK
    Postgres
    ClickHouse
    OpenRouter
    LiteLLM
    Portkey
    Helicone
    API Available
    View Docs

    Ratings & Reviews

    No ratings yet

    Be the first to rate coding.sh and help others make informed decisions.

    Rate other tools you’ve used

    Developer

    Jack Lau

    Founded 2021
    Ames, IA
    Read more about Jack Lau
    WebsiteGitHubLinkedInX / Twitter
    1 tool in directory

    Similar Tools

    Evidently icon

    Evidently

    Open-source Python framework to evaluate, test, and monitor ML and LLM-powered systems and data pipelines.

    Langtrace icon

    Langtrace

    Open-source, OpenTelemetry-based observability and evaluations platform for LLM applications, supporting real-time tracing, metrics, and debugging across popular LLMs, frameworks, and vector databases.

    Traceloop icon

    Traceloop

    Traceloop is an LLM observability and evaluation platform built on OpenTelemetry that turns monitoring and evals into a continuous feedback loop for AI applications.

    Browse all tools

    Related Topics

    Monitoring Tools

    AI-enhanced monitoring solutions that provide real-time visibility into system performance, anomaly detection, and predictive alerting for proactive issue resolution.

    141 tools

    Observability Platforms

    Comprehensive platforms that combine metrics, logs, and traces with AI-powered analytics to provide deep insights into complex distributed systems and application behavior.

    191 tools

    LLM Evaluations

    Platforms and frameworks for evaluating, testing, and benchmarking LLM systems and AI applications. These tools provide evaluators and evaluation models to score AI outputs, measure hallucinations, assess RAG quality, detect failures, and optimize model performance. Features include automated testing with LLM-as-a-judge metrics, component-level evaluation with tracing, regression testing in CI/CD pipelines, custom evaluator creation, dataset curation, and real-time monitoring of production systems. Teams use these solutions to validate prompt effectiveness, compare models side-by-side, ensure answer correctness and relevance, identify bias and toxicity, prevent PII leakage, and continuously improve AI product quality through experiments, benchmarks, and performance analytics.

    243 tools
    Browse all topics
    Back to all toolsSuggest an edit
    ratings
    discussions