Mesh LLM
Mesh LLM is an open-source distributed inference project that pools spare compute across macOS, Linux, Windows and heterogeneous GPU/CPU machines to run open models. It provides a peer-to-peer mesh, local OpenAI-compatible endpoint, model routing, and layer-splitting so users can run models that may not fit on one device.
At a Glance
- Individual developers and open-source users
- Homelabs and enthusiasts with spare compute
- Small teams and businesses with distributed GPU capacity
- Researchers experimenting with distributed inference
- +3 more
AI Tools by Mesh LLM
(1)
Mesh LLM
Distributed LLM Inference Runtime
Discussions
No discussions yet
Be the first to start a discussion about Mesh LLM
Latest News
Mesh LLM v0.78.1 released with automatic-tool, wallet-identity and chunked-request admission fixes.
Mesh LLM v0.78.0 released with plugin UI contribution slots, catalog pinning and pinned default plugins.
Mesh LLM v0.77.0 released with live runtime event streaming and structured native runtime event pipeline work.
iroh published a technical deep dive on Mesh LLM's distributed inference architecture and its QUIC/iroh integration.
Products & Services
Apache-2.0 open-source Rust runtime/daemon and command-line tool for starting nodes, serving models, joining public or private meshes, routing requests, and splitting large models across peers.
A best-effort public compute mesh that users can join as a serving node or API-only client; it exposes a local web console and OpenAI-compatible endpoint.
Operator-controlled meshes for trusted teams, families, communities or businesses, with invite-token/private-network and published-mesh workflows.
Catalog of model references and GGUF/layer-package artifacts, including multi-machine packages that let nodes download and serve only assigned layer ranges.
Market Position
Mesh LLM positions itself against centralized inference APIs by keeping models and compute under the operator's control and turning heterogeneous spare machines into one endpoint. Among distributed/local inference projects, its documented differentiators are cross-platform hardware support, decentralized QUIC/iroh networking, public or private meshes, catalog-backed layer packages for large GGUF models, and agent/plugin integrations. Its official comparison identifies exo as a close alternative: exo emphasizes Apple-Silicon/MLX clusters and tensor or pipeline parallelism, while Mesh emphasizes cross-platform staged layer splits, Rust, gossip-based meshes and OpenAI-compatible serving.
Leadership
Founders
Michael Neale (Mic Neale)
Founder and creator of Mesh LLM according to public project coverage. He is a software engineer and technical leader, co-founder of CloudBees, contributor/early participant in the Drools rules project, and previously worked with JBoss and Red Hat; his public GitHub profile identifies him as a CloudBees co-founder. He is also associated with Block's open-source AI work and Goose.
Founding Story
Mesh LLM was started to let people and teams reuse and pool idle hardware rather than send prompts to centralized, metered inference providers or buy a larger GPU. The initial vision was a private or public peer-to-peer inference cloud in which laptops, workstations, GPU rigs and servers could act as one OpenAI-compatible endpoint, including distributing models across nodes when no single machine can hold them.
Business Model
Revenue Model
The project is distributed as open-source software under the Apache-2.0 license and the public materials describe self-hosted, community and embedded use rather than a paid hosted API or subscription business model.
Target Markets
- Individual developers and open-source users
- Homelabs and enthusiasts with spare compute
- Small teams and businesses with distributed GPU capacity
- Researchers experimenting with distributed inference
- Agent and coding-tool developers
- Communities or trusted groups sharing compute
- Running open-weight LLMs locally without a centralized inference provider
- Pooling spare GPUs and memory across a homelab, office, family or trusted team
- Serving models that exceed the memory of a single laptop or GPU
- Providing a single local endpoint to coding agents and OpenAI-compatible applications
- Experimenting with distributed inference, mixture-of-agents routing and agent collaboration
- Sharing best-effort idle inference capacity through a public mesh