# Razer AIKit

> Open-source AI development toolkit for running and fine-tuning LLMs locally on a single GPU or multi-GPU cluster, built on vLLM, Ray, and LlamaFactory.

Razer AIKit is an open-source AI development toolkit released by Razer AI under the Apache License 2.0. It enables engineers and researchers to run and fine-tune large language models locally on NVIDIA GPUs — from a single consumer GPU to a distributed multi-GPU cluster — without cloud dependency. The project is currently in preview (latest release v0.7.0) and is actively maintained on GitHub.

## What It Is

Razer AIKit is a local-first AI development environment that packages production-grade inference, fine-tuning, and multi-GPU orchestration into a Docker-based toolkit. It is designed for out-of-the-box setup on NVIDIA-accelerated hardware, supporting both x86-64 and ARM64 architectures. The toolkit exposes an OpenAI-compatible API, making it straightforward to connect to existing tools and workflows without rewriting integrations.

## Technical Stack

The toolkit is built on three core open-source components:

- **vLLM** — production-grade LLM inference engine with memory optimization
- **LlamaFactory** — parameter-efficient fine-tuning framework supporting LoRA and other PEFT methods
- **Ray** — distributed computing layer for seamless multi-GPU and multi-node scaling
- **AIKit CLI** (`rzr-aikit`) — command-line interface for model lifecycle management

The full advanced stack also includes Jupyter Lab for interactive notebooks, Grafana for GPU and cluster metrics, Open WebUI for chat-based model testing, and Prometheus for metrics collection.

## Platform and Hardware Support

Razer AIKit targets NVIDIA GPUs with Compute Capability 7.0 (Volta) or higher. Supported hardware spans consumer gaming laptops (Razer Blade with GeForce RTX 20/30/40/50 series), professional workstations (NVIDIA RTX PRO 6000 Blackwell, RTX 6000 Ada), and data center systems (NVIDIA DGX GB10/GB200/GB300, GH200, H100, H200, A100, B200, B300, L4, L40, L40S). The toolkit runs on Windows 11 via WSL 2 and natively on Ubuntu 22.04/24.04. According to the project page, Razer AIKit is optimized and tested on Razer devices but is not limited to Razer hardware.

## Key Capabilities

- Run any of 300,000+ vLLM-compatible models from Hugging Face Hub locally
- AI image generation with iterative local inference and zero cloud dependency
- Fine-tuning with LoRA via LlamaFactory notebooks
- Retrieval-augmented generation (RAG) with Open WebUI
- Semantic search via included notebook examples
- Live GPU load monitoring in Grafana with seamless scale-out to multi-GPU clusters
- OpenAI API compatibility enabling connections to Open WebUI, Continue Coding Assistant, AnythingLLM, and Microsoft AI Dev Gallery

## Update: v0.7.0 Preview and Omni-Modal Expansion

The latest GitHub release is v0.7.0 (published September 11, 2026), still labeled as a preview release. A Razer newsroom post from April 2026 announces that Razer AIKit expanded to omni-modal AI and ARM64 architectures, broadening support beyond text and image generation to additional modalities and to ARM-based data center hardware such as NVIDIA DGX Spark (GB10). The repository was created in September 2025 and has seen continuous pushes through September 2026, indicating active development momentum.

## Deployment Model and Setup Path

Razer AIKit is distributed as a Docker image (`razerofficial/aikit`) available on Docker Hub. The quick-start path requires Docker Engine, an NVIDIA GPU driver, and the NVIDIA Container Toolkit. A single `docker run` command pulls the image and drops users into an interactive environment with Jupyter Lab notebooks and the `rzr-aikit` CLI. The advanced mode uses Docker Compose to bring up the full monitoring and UI stack. The project page also documents a production case study where AIKit ran on a globally distributed pool of consumer GPUs via Akash Network's decentralized compute marketplace during the Razer AVA Mini campaign.

## Features
- Local LLM inference on NVIDIA GPUs
- Multi-GPU cluster scaling via Ray
- Fine-tuning with LoRA via LlamaFactory
- AI image generation (local, no cloud)
- OpenAI-compatible API
- 300,000+ Hugging Face model support
- Jupyter Lab interactive notebooks
- Grafana GPU and cluster monitoring
- Open WebUI chat interface
- Retrieval-augmented generation (RAG)
- Semantic search
- AIKit CLI for model lifecycle management
- Docker-based deployment
- ARM64 architecture support
- Omni-modal AI support

## Integrations
Hugging Face Hub, vLLM, Ray, LlamaFactory, Open WebUI, Grafana, Prometheus, Jupyter Lab, Continue Coding Assistant, AnythingLLM, Microsoft AI Dev Gallery, Docker, Akash Network, NVIDIA CUDA, NVIDIA Container Toolkit

## Platforms
LINUX, WINDOWS, CLI, API

## Pricing
Open Source

## Version
v0.7.0

## Links
- Website: https://www.razer.ai/aikit/
- Documentation: https://doc.wyvrn.com/docs/razer-ai-gaming-tools/
- Repository: https://github.com/razerofficial/aikit
- EveryDev.ai: https://www.everydev.ai/tools/razer-aikit