Baseten

Baseten is an AI inference platform for running open-source, custom, and fine-tuned models in production. It offers dedicated deployments, pre-optimized Model APIs, and a training product, all built on what it calls the Baseten Inference Stack. The company says it was founded in 2019 by engineers.

What It Is

Baseten provides infrastructure for serving AI models at scale. Teams deploy a custom or proprietary model, or use hosted open models through Model APIs, and the platform handles runtime optimization, autoscaling, and deployment management. Supported workloads named on the site include LLMs, transcription, text-to-speech, image generation (including ComfyUI workflows), embeddings, and compound AI via Baseten Chains.

Deployment Options

Workloads can run on Baseten Cloud, a multi-cloud, multi-region network with optional single-tenant clusters, or self-hosted inside a customer's own VPC. A hybrid option adds on-demand flex capacity on Baseten Cloud, and the self-hosted setup can fail over to Baseten Cloud. Baseten also offers forward deployed engineers who work with customers to tune deployments toward latency, throughput, and cost targets.

Performance and Runtimes

Baseten says its runtimes include custom kernels, speculative decoding techniques, and advanced caching. It claims 99.99% uptime and fast cold starts, and says Baseten Embeddings Inference delivers higher throughput and lower latency than other solutions. These are vendor claims.

Security and Enterprise Controls

The enterprise page states Baseten does not store inference inputs or outputs, is SOC 2 Type II certified and HIPAA compliant, and supports region-restricted deployments for data residency. It lists RBAC, team-level access controls, SSO, logging, custom metrics, and tracing.

Training and Model Labs

The Training product uses the Loops SDK for reinforcement-learning-style training, with deployment to production inference on the same stack. Baseten for Model Labs lets model creators distribute and monetize models on Baseten infrastructure.