EveryDev.ai
Subscribe
Home
Tools

5,455+ AI tools

  • Featured
  • Gems
  • Trending
  • New
  • Rate tools
  • Compare
  • Arena
Categories
  • Agents2765
  • Coding2061
  • Infrastructure976
  • Projects768
  • Research694
  • MCP675
  • Analytics530
  • Marketing508
  • Design446
  • Testing439
  • Data410
  • Security372
  • Integration273
  • Prompts262
  • Communication260
  • Extensions228
  • Voice186
  • Commerce177
  • DevOps177
  • Web124
  • Learning116
  • Finance40
AI Tools by Topic
  • AI Coding Assistants
  • Agent Frameworks
  • MCP Servers
  • AI Prompt Tools
  • Vibe Coding Tools
  • AI Design Tools
  • AI Database Tools
  • AI Website Builders
  • AI Testing Tools
  • LLM Evaluations
Follow Us
  • X / Twitter
  • LinkedIn
  • Reddit
  • Discord
  • Threads
  • Bluesky
  • Mastodon
  • YouTube
  • GitHub
  • Instagram
Get Started
  • Users
  • Rate Tools
  • About
  • Editorial Standards
  • Corrections & Disclosures
  • Community Guidelines
  • Advertise
  • Contact Us
  • Newsletter
  • Submit a Tool
  • Start a Discussion
  • Write A Blog
  • Share A Build
  • Terms of Service
  • Privacy Policy
Explore with AI
  • ChatGPT
  • Gemini
  • Claude
  • Grok
  • Perplexity
Agent Experience
  • llms.txt
Theme
With AI, Everyone is a Dev. EveryDev.ai ©
    1. Home
    2. Tools
    3. Oumi
    Oumi icon

    Oumi

    LLM Evaluations
    Featured

    Open-source platform and commercial AI factory for fine-tuning, evaluating, deploying, and continuously improving custom open foundation models.

    Visit Website

    At a Glance

    Pricing
    Open Source
    Free tier available

    Self-hosted Oumi open-source core under Apache License 2.0.

    Pro: $25/mo
    Enterprise: Custom/contact

    Engagement

    Available On

    CLI
    API
    SDK
    Web
    Linux

    Resources

    WebsiteDocsGitHubllms.txt

    Topics

    LLM EvaluationsAI InfrastructureModel Management

    Alternatives

    SilicoFinetuneDBXinference
    Developer
    OumiOumi PBC builds an open-source stack and commercial platform…

    Listed Nov 2026

    About Oumi

    Oumi, made by Oumi PBC, is an open-source stack and hosted platform for building specialized AI models. The open-source repository is licensed under Apache 2.0 and covers training, fine-tuning, evaluation, inference, and data synthesis driven by YAML configs. The company's platform adds an agent-guided workflow from a plain-English task description to a deployed model.

    What It Is

    Oumi OSS is aimed at ML engineers and researchers who want one interface for the model lifecycle instead of stitching together separate libraries. It supports training and fine-tuning from small models to 400B+ parameters using SFT, LoRA, QLoRA, DPO, GRPO and other methods. It works with text and multimodal models, including Llama, Gemma, Qwen, DeepSeek, GLM, Phi and gpt-oss families. The README notes the project is in beta, with stable core features.

    Workflow and Outputs

    The workflow runs from a prompt through evaluation, data synthesis, training, deployment and ongoing retraining. Evaluation covers standard benchmarks, LLM judges and multi-criteria rubric judges. Data synthesis and curation can use LLM judges and include multi-turn tool-use conversations. Inference runs on engines such as vLLM, SGLang and llama.cpp or through commercial APIs, and oumi deploy targets dedicated inference endpoints. Jobs can be launched on AWS, GCP, Azure, Lambda and other clouds with oumi launch.

    Agentic Model Training

    The v0.9 release, per the README, is an end-to-end stack for agentic, tool-using models. It adds tool-calling SFT/DPO, executable tool environments (database, HTTP endpoint, lookup, simulated), and GRPO over Oumi environments with verl. An oumi-mcp MCP server lets Claude or Cursor drive Oumi.

    Platform and Ownership Claims

    Oumi's website says users own the resulting weights, data and recipes and can run them on Oumi's GPUs or in their own infrastructure. The company states that specialized models can cost up to 90% less than frontier models and reach up to 50% higher accuracy; these are vendor claims. It also lists a continual-improvement loop in which production failures feed retraining, and says deployments scale GPUs to zero when idle.

    Oumi - 1

    Community Discussions

    Be the first to start a conversation about Oumi

    Share your experience with Oumi, ask questions, or help others learn from your insights.

    Pricing

    OPEN SOURCE

    Open Source (Oumi core)

    Self-hosted Oumi open-source core under Apache License 2.0.

    • Apache License 2.0
    • Train/fine-tune, evaluate, synthesize data, and serve models from the open-source stack
    • Run anywhere: laptops, clusters, clouds
    FREE

    Free

    Hosted platform free tier for exploring and prototyping.

    • Up to $50 in credits for 3 months
    • Full automation with Oumi agent (Pat. Pend.)
    • Data synthesis with open/closed models
    • Data analysis and curation
    • Open/closed model evaluation with failure modes

    Pro

    For teams and builders; pay-as-you-go after the included credits.

    $25
    per month
    • Everything in Free
    • Training with on-policy distillation
    • Multiple concurrent jobs for higher efficiency
    • Download model weights: 1 / month
    • Production inference with autoscaling
    • $25/month credits platform-wide
    • Pay-as-you-go after credits

    Enterprise

    For production at scale with enterprise-grade security and controls.

    Custom
    contact sales
    • Everything in Pro
    • BYOC - VPC & on-prem
    • Dedicated capacity and autoscaling to 1000s of GPUs
    • Guaranteed throughput and SLAs
    • Advanced training methods (RL, custom pipelines)
    • Dedicated experts embedded with your team
    View official pricing

    Capabilities

    Key Features

    • Fine-tuning with SFT, LoRA, QLoRA, DPO, GRPO
    • Tool-calling and agentic model training with executable environments
    • Synthetic data generation and curation with LLM judges
    • Benchmark and rubric-based model evaluation
    • Inference with vLLM, SGLang and llama.cpp
    • One-command deployment to dedicated endpoints (oumi deploy)
    • Remote job launch on AWS, GCP, Azure, Lambda
    • Hyperparameter tuning
    • YAML-based recipes
    • MCP server for Claude and Cursor
    • Model distillation
    • Continual retraining from production signals

    Integrations

    Hugging Face Transformers
    vLLM
    SGLang
    llama.cpp
    verl
    OpenAI
    Anthropic
    Vertex AI
    Together
    Fireworks
    Parasail
    OpenRouter
    AWS
    GCP
    Azure
    Lambda
    Modal
    Slurm
    Docker
    Claude
    Cursor
    OpenEnv
    API Available
    View Docs

    Demo Video

    Oumi Demo Video
    Watch on YouTube

    Ratings & Reviews

    No ratings yet

    Be the first to rate Oumi and help others make informed decisions.

    Rate other tools you’ve used

    Developer

    Oumi Team

    Oumi PBC builds an open-source stack and commercial platform for creating, deploying and improving custom AI models. The company was cofounded by Manos Koukoumidis, Matthew Persons, Jeremiah Greer, William Zeng and Kostas Aisopos, who previously worked at Google. It says its mission is to make frontier AI open and accessible to all, and announced a $10M seed round in January 2025.

    Read more about Oumi Team
    WebsiteGitHubLinkedInX / Twitter
    1 tool in directory

    Similar Tools

    Silico icon

    Silico

    Silico is a platform by Goodfire for building and debugging AI models with interpretability-driven precision, letting teams see what models have learned and make targeted interventions.

    FinetuneDB icon

    FinetuneDB

    AI fine-tuning platform to create custom LLMs by training models with your data in minutes, not weeks.

    Xinference icon

    Xinference

    Open-source platform for serving LLM, embedding, image, audio, video and multimodal models through a unified OpenAI-compatible API.

    Browse all tools

    Related Topics

    LLM Evaluations

    Platforms and frameworks for evaluating, testing, and benchmarking LLM systems and AI applications. These tools provide evaluators and evaluation models to score AI outputs, measure hallucinations, assess RAG quality, detect failures, and optimize model performance. Features include automated testing with LLM-as-a-judge metrics, component-level evaluation with tracing, regression testing in CI/CD pipelines, custom evaluator creation, dataset curation, and real-time monitoring of production systems. Teams use these solutions to validate prompt effectiveness, compare models side-by-side, ensure answer correctness and relevance, identify bias and toxicity, prevent PII leakage, and continuously improve AI product quality through experiments, benchmarks, and performance analytics.

    227 tools

    AI Infrastructure

    Infrastructure designed for deploying and running AI models.

    560 tools

    Model Management

    Tools for managing, versioning, and deploying AI models.

    133 tools
    Browse all topics
    Back to all toolsSuggest an edit
    ratings
    discussions