EveryDev.ai
Subscribe
Home
Tools

5,901+ AI tools

  • Featured
  • Gems
  • Trending
  • New
  • Rate tools
  • Compare
  • Arena
Categories
  • Agents2972
  • Coding2248
  • Infrastructure1067
  • Projects818
  • MCP794
  • Research744
  • Analytics565
  • Marketing527
  • Design488
  • Testing476
  • Data442
  • Security412
  • Integration288
  • Communication284
  • Prompts283
  • Extensions238
  • Voice213
  • DevOps190
  • Commerce183
  • Web128
  • Learning123
  • Finance44
AI Tools by Topic
  • AI Coding Assistants
  • Agent Frameworks
  • MCP Servers
  • AI Prompt Tools
  • Vibe Coding Tools
  • AI Design Tools
  • AI Database Tools
  • AI Website Builders
  • AI Testing Tools
  • LLM Evaluations
Follow Us
  • X / Twitter
  • LinkedIn
  • Reddit
  • Discord
  • Threads
  • Bluesky
  • Mastodon
  • YouTube
  • GitHub
  • Instagram
Get Started
  • Users
  • Rate Tools
  • About
  • Editorial Standards
  • Corrections & Disclosures
  • Community Guidelines
  • Advertise
  • Contact Us
  • Newsletter
  • Submit a Tool
  • Start a Discussion
  • Write A Blog
  • Share A Build
  • Terms of Service
  • Privacy Policy
Explore with AI
  • ChatGPT
  • Gemini
  • Claude
  • Grok
  • Perplexity
Agent Experience
  • llms.txt
Theme
With AI, Everyone is a Dev. EveryDev.ai ©
    1. Home
    2. Tools
    3. Distributed Llama
    Distributed Llama icon

    Distributed Llama

    AI Infrastructure
    Featured

    Open-source C++ tool that connects multiple home devices into a cluster to accelerate LLM inference using tensor parallelism over Ethernet.

    Visit Website

    At a Glance

    Pricing
    Open Source

    Distributed Llama core software released under the MIT license; free to use, copy, modify, and distribute.

    Engagement

    Available On

    Linux
    macOS
    Windows
    CLI
    API

    Resources

    WebsiteDocsGitHubllms.txt

    Topics

    AI InfrastructureCompute OptimizationLocal Inference

    Alternatives

    StrataTensorFoldMesh LLM
    Developer
    Bart Tadych

    Listed Nov 2026

    About Distributed Llama

    Distributed Llama is an open-source project by Bartłomiej Tadych (b4rtaz) that links several ordinary devices into a cluster for running large language models. It splits the neural network across machines using tensor parallelism and synchronizes state over Ethernet. It runs on Linux, macOS and Windows and is optimized for ARM and x86_64 AVX2 CPUs. The latest listed release is v0.17.0.

    What It Is

    Distributed Llama is a local, distributed LLM inference engine written in C++. One root node loads the model and weights and sends them to worker nodes, then keeps the network state synchronized. The root node also processes its own slice of the network. Worker nodes process their slices and need no model-specific configuration. RAM usage is split across all nodes, with the root node needing slightly more than workers.

    How the Workflow Runs

    A single launch.py command downloads a model and tokenizer and sets up the root node; this needs Python 3 and a C++ compiler. Documented models include Llama 3.1, 3.2 and 3.3, DeepSeek R1 Distill Llama 8B, and several Qwen 3 and Qwen 3.5 variants. Hugging Face models can also be converted manually. The CLI offers dllama inference (with a simple benchmark), dllama chat, dllama worker, and dllama-api to start an API server. Arguments control model path, tokenizer, worker addresses, thread count, maximum sequence length and sync precision. Guides cover Raspberry Pi and GPU setups, and the README notes experimental Vulkan support.

    Known Limitations

    • The node count must be a power of two (1, 2, 4, ...).
    • The maximum number of nodes equals the number of KV heads in the model.
    • Only q40 models with q80 buffer-float-type and f32 models with f32 buffer-float-type are supported.
    Distributed Llama - 1

    Community Discussions

    Be the first to start a conversation about Distributed Llama

    Share your experience with Distributed Llama, ask questions, or help others learn from your insights.

    Pricing

    OPEN SOURCE

    Open Source (MIT)

    Distributed Llama core software released under the MIT license; free to use, copy, modify, and distribute.

    • MIT License
    • Connect home devices into a cluster to accelerate LLM inference
    • Supports Linux, macOS, and Windows
    • Optimized for ARM and x86_64 AVX2 CPUs
    • Commands: dllama inference, chat, worker, dllama-api

    Capabilities

    Key Features

    • Tensor-parallel LLM inference across multiple devices
    • Root and worker node architecture
    • Single-command model download and setup via launch.py
    • CLI chat and inference benchmark
    • API server (dllama-api)
    • Hugging Face model conversion
    • Experimental Vulkan GPU support
    • Optimized for ARM and x86_64 AVX2 CPUs
    • Raspberry Pi support

    Integrations

    Hugging Face
    Vulkan
    API Available
    View Docs

    Ratings & Reviews

    No ratings yet

    Be the first to rate Distributed Llama and help others make informed decisions.

    Rate other tools you’ve used

    Developer

    Bart Tadych

    Read more about Bart Tadych
    WebsiteGitHubX / Twitter
    1 tool in directory

    Similar Tools

    Strata icon

    Strata

    Open-source inference engine that runs the Qwen3.8-Flash-Next model locally on consumer NVIDIA or AMD GPUs with an OpenAI/Anthropic-compatible local API.

    TensorFold icon

    TensorFold

    Open-source local LLM inference engine written in Zig that serves open models on Apple Silicon (Metal) and NVIDIA GPUs (CUDA) through OpenAI- and Anthropic-compatible APIs.

    Mesh LLM icon

    Mesh LLM

    Open-source distributed LLM inference that pools GPUs across machines and serves them through one OpenAI-compatible API.

    Browse all tools

    Related Topics

    AI Infrastructure

    Infrastructure designed for deploying and running AI models.

    589 tools

    Compute Optimization

    Tools for optimizing computational resources and performance.

    90 tools

    Local Inference

    Tools and platforms for running AI inference locally without cloud dependence.

    353 tools
    Browse all topics
    Back to all toolsSuggest an edit
    ratings
    discussions
    5views