# Universal LLM Client

> A provider-agnostic TypeScript/JavaScript LLM client with transparent failover across OpenAI, Anthropic, Google Gemini, Ollama, and any OpenAI-compatible endpoint.

Universal LLM Client is an open-source TypeScript/JavaScript library that gives developers a single `AIModel` class to interact with any major LLM provider — OpenAI, Anthropic, Google Gemini, Ollama, Vertex AI, LlamaCpp, and any OpenAI-compatible endpoint. It is MIT-licensed, zero-dependency at its core, and designed to run on Node.js 22+, Bun, Deno, and modern browsers. The project is maintained by Igor Lins e Silva and published on npm as `universal-llm-client`.

## What It Is

Universal LLM Client is a provider-agnostic reasoning and chat API for JavaScript/TypeScript. Rather than writing separate integration code for each LLM backend, developers configure a priority-ordered list of providers and call a single unified API. The library handles failover, retries, health tracking, streaming, structured output, tool calling, and observability behind the scenes — the calling code never needs to know which backend actually served the response.

## Transparent Failover Architecture

The core differentiator is the built-in Router/failover engine. Providers are assigned priorities; if the highest-priority provider returns an error or times out, the library retries (configurable, default 2 retries) and then seamlessly moves to the next provider in the chain. Health tracking and cooldowns prevent repeated calls to a known-bad backend. The `getProviderStatus()` method exposes live health state at any time.

- Supports Google AI Studio → OpenRouter → local Ollama chains out of the box
- Configurable `retries` and `timeout` per model instance
- Failover events are captured by the built-in Auditor for observability

## Unified Reasoning and Streaming

The library exposes a single `thinking` flag — `true`/`false` or a level string (`'minimal' | 'low' | 'medium' | 'high'`) — that maps to each provider's native reasoning control: Gemini `thinkingLevel`/`thinkingBudget`, OpenAI `reasoning_effort`, Anthropic `budget_tokens`, Ollama `think`, and vLLM `enable_thinking`. Chain-of-thought is surfaced as `response.reasoning` and as streaming `thinking` events. Streaming uses async generators with pluggable decoder strategies — standard chat, interleaved reasoning (parses `<think>` tags), and passthrough — selectable per call.

## Tool Calling, MCP, and Structured Output

Tools are registered once via `registerTool()` or a fluent `ToolBuilder` and work across all providers. The `chatWithTools()` method runs an autonomous multi-turn execution loop and returns a full tool execution trace. MCP (Model Context Protocol) servers are bridged via `MCPToolBridge`, supporting both stdio and HTTP transports with automatic tool discovery — no glue code required. Structured output uses Zod 4 schemas: `generateStructured()` returns fully typed, validated JSON; `tryParseStructured()` is a non-throwing variant; and `generateStructuredStream()` yields partial validated objects as the response streams in.

## Agent Framework Design

The library is explicitly designed as a transport layer for agent frameworks. Key design choices include:

- **Stateless** — no conversation history stored; the framework manages memory
- **Composable** — separate instances for chat, embeddings, and vision
- **Observable** — every request, response, retry, failover, and tool call is a structured, flushable `AuditEvent` via the `Auditor` interface (built-in `ConsoleAuditor` and `BufferedAuditor` for custom sinks like OpenTelemetry)
- **Context budget** — `getModelInfo()` exposes `contextLength` for budget management
- **Embeddings** — single and batch embedding generation via `embed()` and `embedArray()`

## Update: v4.6.0 — Pool-Aware Routing and Failover Hardening

The latest release, v4.6.0 (published July 22, 2026), is titled "pool-aware routing, StreamLoopGuard & failover hardening." The repository was created in July 2025 and has seen active development through mid-2026, with the last push on August 3, 2026. The project is in active development with a versioned release cadence on npm.

## Features
- Transparent provider failover with retries and health tracking
- Unified reasoning API (thinking flag mapped to each backend)
- Streaming via async generators with pluggable decoder strategies
- Autonomous tool calling with multi-turn execution loop
- MCP server integration via MCPToolBridge (stdio and HTTP)
- Structured output with Zod 4 schemas and streaming partial objects
- Gemini Deep Research support (background interactions with polling)
- Built-in observability via Auditor interface (ConsoleAuditor, BufferedAuditor)
- Single and batch embedding generation
- Multimodal/vision support
- Zero core dependencies
- Runs on Node.js 22+, Bun, Deno, and modern browsers
- Provider health status API
- ToolBuilder fluent API and ToolExecutor wrappers
- Tree-shakeable structured output module

## Integrations
OpenAI, Anthropic Claude, Google Gemini (AI Studio), Google Vertex AI, Ollama, LlamaCpp, OpenRouter, xAI/Grok, Mistral, DeepSeek, Groq, Together AI, Fireworks AI, Perplexity Sonar, vLLM, LM Studio, Azure OpenAI, Model Context Protocol (MCP), OpenTelemetry, Zod

## Platforms
API, DEVELOPER_SDK, CLI

## Pricing
Open Source

## Version
v4.6.0

## Links
- Website: https://igorls.github.io/universal-llm-client/
- Documentation: https://igorls.github.io/universal-llm-client/guide/getting-started.html
- Repository: https://github.com/igorls/universal-llm-client
- EveryDev.ai: https://www.everydev.ai/tools/universal-llm-client