VernLLM

VernLLM is an open-source TypeScript library that wraps your existing LLM provider clients with a production-grade resilience layer. It runs entirely in your own process — no extra network hop, no proxy — and ships with zero runtime dependencies. The latest release is vern-llm@2.8.0, published in September 2026.

What It Is

VernLLM is an LLM call framework that sits between your application code and provider SDKs (OpenAI, Anthropic, Gemini, AWS Bedrock, and dozens of OpenAI-compatible endpoints). Rather than replacing your provider client, it wraps it through a small adapter interface and adds configurable resilience primitives: retries with backoff and jitter, a circuit breaker, multi-provider fallback, local rate limiting, response caching, and request/response middleware. The core export is a single VernLLM class that accepts all options at construction time and exposes a typed call / cachedCall API.

Core Resilience Primitives

Each primitive is independently configurable and can be replaced with a custom implementation:

• Retry budget — caps what fraction of recent traffic can be retries within a rolling time window, preventing retry storms
• Circuit breaker — stops cascading failures by halting calls to a provider that is repeatedly failing
• Provider fallback — automatically routes to a backup provider (e.g., Anthropic when OpenAI is down) within the same process
• Rate limiting — queues requests locally under per-minute request, token, and concurrency ceilings
• Timeout — prevents hanging calls from blocking indefinitely
• Caching — cachedCall accepts any adapter implementing get/set, so Redis, a database, or a custom store can replace the built-in in-memory cache
• Middleware — transform edits or redacts outgoing requests; wrap runs around the full logical call including retries and fallback attempts, enabling logging, tracing, or cost tracking

Provider Coverage

VernLLM ships first-class adapters for OpenAI (fromOpenAI), Anthropic (fromAnthropic), Google Gemini (fromGemini), and AWS Bedrock. Any OpenAI-compatible endpoint — including Groq, Mistral, DeepSeek, Cerebras, Together AI, Fireworks AI, Ollama, OpenRouter, Perplexity, DeepInfra, LM Studio, vLLM, xAI (Grok), NVIDIA NIM, Vercel AI Gateway, Cloudflare Workers AI, SambaNova, Lambda Labs, Snowflake Cortex, and more — is supported through a shared fromOpenAI-compatible path. A fromFetch adapter covers any provider reachable over HTTP.

Architecture and Tradeoffs

VernLLM is deliberately in-process rather than a gateway. The homepage explains the tradeoff directly: a gateway is the better choice for one shared setup across many services or languages, while VernLLM is the better choice when you want to react to your own application logic, avoid an extra network hop, or keep the resilience layer close to the call site. The library is written in TypeScript from the ground up; structured output schemas, call parameters, and errors are all typed so mistakes surface at compile time. It does not bundle Zod or any provider SDK — you bring your own clients and schema validators.

Update: vern-llm@2.8.0

The repository was created in July 2026 and has been actively developed, with the latest release (vern-llm@2.8.0) published on September 8, 2026. The project is a pnpm monorepo with two workspaces: the packages/vern-llm npm package and an apps/docs Fumadocs-powered documentation site at vernllm.dev. GitHub CI workflows cover build checks, linting, type checking, unit tests, integration tests, CodeQL analysis, and code coverage via Codecov. The project carries an OpenSSF Best Practices badge and is licensed MIT.