SPECTRA
Calibration-free 4-bit LLM compression engine and Triton-based GPU serving runtime for running models on consumer and edge GPUs.
At a Glance
Royalty-free use for personal study, scientific reproduction, independent verification, and non-commercial academic research under the SPECTRA Research & Evaluation License.
Engagement
Available On
Alternatives
Listed Nov 2026
About SPECTRA
SPECTRA (Sparse-Precision Exact Channel Tensor Reconstruction Architecture) is a model compression framework and local inference runtime from NeuraCryption, published with a research paper archived on Zenodo. It compresses Hugging Face models into a 4-bit container format and serves them on NVIDIA GPUs, targeting laptops and desktops with limited VRAM. The source code is released under a non-commercial research and evaluation license.
What It Is
SPECTRA takes an existing language model, including fine-tuned ones, and converts it into a 4-bit .trdc container without any calibration dataset. It then loads that container into GPU memory and runs generation through custom fused Triton kernels. It supports dense Transformers, Mamba-2 hybrids and convolutional/state-space hybrids.
How the Pipeline Works
The compressor reads the model's normalization scale parameters to estimate channel sensitivity analytically, so no calibration text is needed. Weights are quantized in Group-32 blocks with per-group scales and zero points, with optional FP16 outlier channels kept separate. Tensors are streamed from disk shard by shard, which the README says keeps host RAM under 50 MB during compression. The resulting TRDC container is memory-mapped and streamed into GPU buffers.
At runtime, fused W4A16 kernels handle single-token decoding and batched Tensor Core kernels handle prefill. Supporting components include a fused embedding that dequantizes only the active vocabulary rows, a double-buffered layer offloader for models that exceed VRAM, and a 4-bit KV cache.
Ways to Use It
The repository ships a launcher script, a CLI for compress, run and status commands, a Python SDK, a local web dashboard with hardware telemetry and streaming chat, and REST endpoints for loading models, generating text and compressing. The paper's appendix also describes an OpenAI-compatible local server.
Reported Results
The NeuraCryption paper reports 2.80x to 3.31x compression across six models on an RTX 3050 Ti laptop GPU with 4 GB VRAM. It reports 82.53 tokens per second for DeepSeek-R1-Distill-1.5B and 35.58% ARC-Challenge accuracy versus 34.73% for the FP16 baseline. These are the authors' own benchmarks.
Licensing
The license permits personal study, academic research and non-commercial evaluation. Commercial use requires a separate written agreement with NeuraCryption.
Community Discussions
Be the first to start a conversation about SPECTRA
Share your experience with SPECTRA, ask questions, or help others learn from your insights.
Pricing
Non-Commercial Research
Royalty-free use for personal study, scientific reproduction, independent verification, and non-commercial academic research under the SPECTRA Research & Evaluation License.
- Access to the reference implementation
- Non-commercial research and peer-review benchmarking
- Subject to NeuraCryption's license and governing terms
Commercial License
Commercial deployment or institutional distribution requires a separate written agreement with NeuraCryption. Prices, billing currency, and contract terms are negotiated.
- Commercial rights defined by written agreement
Capabilities
Key Features
- Calibration-free 4-bit Group-32 microscaling quantization
- Analytical channel saliency from normalization scales
- Adaptive FP16 outlier channel isolation
- TRDC4B01 binary container with memory-mapped loading
- Fused W4A16 Triton GEMV and batched Tensor Core GEMM kernels
- Fused embedding with dynamic row-gather dequantization
- Ping-pong layer offloading over PCIe
- 4-bit quantized KV cache
- Hardware probing via NVML with automatic execution planning
- Prompt lookup speculative decoding
- Web dashboard with live telemetry and streaming chat
- CLI, Python SDK and REST API
