wigolo

wigolo is a local-first web intelligence server built for AI agents, created and maintained by Towhid Khan (@KnockOutEZ). It runs entirely on your machine, requires no API keys for its core tools, and charges nothing per query — the expensive work (ranking, embeddings, browser rendering) happens on-device rather than in a metered cloud. The project is licensed under AGPL-3.0, currently in public beta, and available on npm, PyPI, Docker, and the official MCP Registry.

What It Is

wigolo is an MCP server (Model Context Protocol) that exposes ten web tools — search, fetch, crawl, extract, cache, find_similar, research, agent, diff, and watch — to any AI agent that can speak MCP, REST, or use a TypeScript/Python SDK. It is designed as a focused web layer for coding agents and self-hosted agent frameworks, not a hosted SaaS or a general-purpose scraping platform. The core architecture is a single Node.js process (Node ≥ 20) that lazy-loads an in-process browser pool, on-device ML models, and a local keyword + vector cache under ~/.wigolo/.

How the Tool Layer Works

Each of the ten tools covers a distinct part of the web-access problem:

• search — fans a query array across 18 direct search-engine adapters in parallel, applies rank fusion and an on-device ML reranker, and returns every result with a transparent score breakdown (semantic, lexical, engine consensus) and byte-offset source spans.
• fetch — uses a tiered router: plain HTTP first, TLS-impersonation for bot walls, and a headless browser engine for SPA shells or challenge pages. Escalation is based on observable signals (challenge bodies, thin content), not domain guesses, and the router learns per domain.
• crawl — supports BFS, DFS, sitemap, and map-only modes with robots.txt respect and per-domain rate limits.
• extract — pulls structured data: tables, JSON-LD, metadata, brand assets, named schemas (Article, Recipe, Product), or any custom JSON Schema.
• cache — every fetched page lands in a local hybrid keyword + vector index; re-querying is instant and works offline.
• find_similar — 3-way fusion of keyword, semantic, and live web signals.
• research — decomposes a question into sub-queries, fans them out, fetches sources, and synthesizes a cited report (requires an opt-in LLM for prose synthesis; returns a raw brief otherwise).
• agent — autonomous plan → search → fetch → extract → synthesize loop with a step log and time budget.
• diff / watch — change detection on a URL since last visit, with optional webhook delivery.

Deployment Model and Integration Surface

wigolo wires into agents through multiple surfaces from the same local process:

• MCP over stdio — auto-configured for Claude Code, Cursor, Codex, Gemini CLI, VS Code, Windsurf, Zed, and Antigravity via npx wigolo init --agents=<your-agent>.
• REST API — wigolo serve exposes POST /v1/{tool} with an OpenAPI 3.1 contract; also serves remote MCP clients over /mcp and /sse from the same port.
• TypeScript SDK — npm install wigolo-sdk; zero-dependency, works on Node, Bun, Deno, and edge runtimes.
• Python SDK — pip install wigolo; standard library only, sync and async.
• Framework integrations — wigolo-langchain (BaseTool + BaseRetriever), wigolo-crewai, wigolo-llamaindex (BaseReader), wigolo-vercel-ai-sdk.
• Docker — available on GitHub Container Registry (ghcr.io/knockoutez/wigolo) and Docker Hub (towhid69420/wigolo).
• Official MCP Registry — registered as io.github.KnockOutEZ/wigolo.

Privacy and Honesty by Design

All cache, embeddings, models, and config live under ~/.wigolo/ on the user's machine. Nothing reaches a third party unless the user explicitly opts into an LLM provider for synthesis (Gemini, Anthropic, OpenAI, Groq, or a local Ollama instance). The project documents that stale cache, failed fetches, degraded backends, and truncation are surfaced in results rather than disguised as empty-but-successful data — a bot-protected page that cannot be read returns a labeled blocked_by_challenge failure rather than a challenge shell dressed as content.

Update: v0.2.0 (July 2026)

The latest release is v0.2.0, published on July 17, 2026. The GitHub repository was last pushed on July 18, 2026, and the project reports a 7,600-test suite backing the public beta. The project page notes the tool set has grown to ten tools and the search adapter count stands at 18 engines. The maintainer describes the current status as public beta — "everything documented here works" — with the v1 milestone tied to sufficient real-world usage and community feedback. The AGPL-3.0 license is described as a structural commitment against a future closed, hosted fork.