GenieX

GenieX is an on-device Gen AI inference runtime from Qualcomm for running language and vision-language models locally on Snapdragon devices. It targets the Hexagon NPU, Adreno GPU, or CPU and is exposed through a CLI, Python, Kotlin/Java, Docker, and an OpenAI-compatible server. It is the community version of Qualcomm GENIE and is currently in developer preview.

What It Is

GenieX is a local inference runtime built around a single C SDK. Users bring a GGUF model from Hugging Face, ModelScope, or Docker Hub, or use a pre-compiled bundle from Qualcomm AI Hub, and run it on Qualcomm hardware with a few lines of code. It supports both LLMs and VLMs.

Two Runtimes, Two Goals

The SDK dispatches to one of two runtimes. The llama.cpp runtime uses GGML kernels across CPU, GPU, and Hexagon HTP and covers almost any community GGUF model. The Qualcomm AI Engine Direct runtime (qairt) is NPU-only and runs per-chipset bundles from Qualcomm AI Hub, which the docs describe as the path for peak on-device performance. For llama.cpp, the README recommends the Q4_0 precision for the best Hexagon NPU support.

Interfaces and Workflow

• CLI: commands such as geniex infer <model> for chat, geniex pull, and geniex model list.
• Python: pip install geniex, with an API that mirrors Hugging Face transformers (from_pretrained() then .generate()).
• OpenAI-compatible server: geniex serve exposes a local /v1 endpoint for existing OpenAI clients.
• Android: a Kotlin/Java SDK distributed as a Gradle dependency, with a sample chat app in a separate repository.
• Docker: a container image that wraps the CLI.
• C/C++: a single header that the other interfaces wrap.

Supported Platforms

GenieX runs only on Qualcomm Snapdragon: Windows ARM64 (Snapdragon X), Android (Snapdragon 8 Elite), and Linux ARM64 (Dragonwing IoT). Tutorials cover benchmarking with geniex-bench, speculative decoding with MTP, and audio input. The project is released under the BSD 3-Clause License.