Universal LLM Client

Universal LLM Client is an open-source TypeScript/JavaScript library that gives developers a single AIModel class to interact with any major LLM provider — OpenAI, Anthropic, Google Gemini, Ollama, Vertex AI, LlamaCpp, and any OpenAI-compatible endpoint. It is MIT-licensed, zero-dependency at its core, and designed to run on Node.js 22+, Bun, Deno, and modern browsers. The project is maintained by Igor Lins e Silva and published on npm as universal-llm-client.

What It Is

Universal LLM Client is a provider-agnostic reasoning and chat API for JavaScript/TypeScript. Rather than writing separate integration code for each LLM backend, developers configure a priority-ordered list of providers and call a single unified API. The library handles failover, retries, health tracking, streaming, structured output, tool calling, and observability behind the scenes — the calling code never needs to know which backend actually served the response.

Transparent Failover Architecture

The core differentiator is the built-in Router/failover engine. Providers are assigned priorities; if the highest-priority provider returns an error or times out, the library retries (configurable, default 2 retries) and then seamlessly moves to the next provider in the chain. Health tracking and cooldowns prevent repeated calls to a known-bad backend. The getProviderStatus() method exposes live health state at any time.

• Supports Google AI Studio → OpenRouter → local Ollama chains out of the box
• Configurable retries and timeout per model instance
• Failover events are captured by the built-in Auditor for observability

Unified Reasoning and Streaming

The library exposes a single thinking flag — true/false or a level string ('minimal' | 'low' | 'medium' | 'high') — that maps to each provider's native reasoning control: Gemini thinkingLevel/thinkingBudget, OpenAI reasoning_effort, Anthropic budget_tokens, Ollama think, and vLLM enable_thinking. Chain-of-thought is surfaced as response.reasoning and as streaming thinking events. Streaming uses async generators with pluggable decoder strategies — standard chat, interleaved reasoning (parses <think> tags), and passthrough — selectable per call.

Tool Calling, MCP, and Structured Output

Tools are registered once via registerTool() or a fluent ToolBuilder and work across all providers. The chatWithTools() method runs an autonomous multi-turn execution loop and returns a full tool execution trace. MCP (Model Context Protocol) servers are bridged via MCPToolBridge, supporting both stdio and HTTP transports with automatic tool discovery — no glue code required. Structured output uses Zod 4 schemas: generateStructured() returns fully typed, validated JSON; tryParseStructured() is a non-throwing variant; and generateStructuredStream() yields partial validated objects as the response streams in.

Agent Framework Design

The library is explicitly designed as a transport layer for agent frameworks. Key design choices include:

• Stateless — no conversation history stored; the framework manages memory
• Composable — separate instances for chat, embeddings, and vision
• Observable — every request, response, retry, failover, and tool call is a structured, flushable AuditEvent via the Auditor interface (built-in ConsoleAuditor and BufferedAuditor for custom sinks like OpenTelemetry)
• Context budget — getModelInfo() exposes contextLength for budget management
• Embeddings — single and batch embedding generation via embed() and embedArray()

Update: v4.6.0 — Pool-Aware Routing and Failover Hardening

The latest release, v4.6.0 (published July 22, 2026), is titled "pool-aware routing, StreamLoopGuard & failover hardening." The repository was created in July 2025 and has seen active development through mid-2026, with the last push on August 3, 2026. The project is in active development with a versioned release cadence on npm.