EAGLE-4

EAGLE-4 is the latest generation of the EAGLE family from Eagle AI Corp for accelerating large language model inference through speculative decoding. The repository ships eagle-harness, a pip-installable package that serves EAGLE-4 drafts on SGLang, vLLM and the experimental TokenSpeed engine. It is source-available software under the Eagle 4 Source-Available License, not an OSI-approved open-source license.

What It Is

EAGLE-4 (Extrapolation Algorithm for Greater Language-model Efficiency) is a drafter that adapts its speculative decoding strategy to batch size, hardware and workload while keeping the target model's output distribution. The draft uses a sliding-window draft layer, a low-rank candidate head and an append-only draft cache. Dense and latent-MoE draft structures are implemented, and Qwen3.8-27B with its EAGLE-4 draft is the measured target pair.

How It Runs

The eagle-harness package registers through hooks the inference engine exposes and does not patch the engine at runtime. Patched engines are distributed as pinned wheels and container images. An eagle-harness-check command verifies that the installed engine provides the hooks the plugin needs. On SGLang, the tree is sized from costs measured during served rounds, while vLLM uses a static tree whose depth stops on calibrated costs produced with a calibration command.

Supported Hardware and Limits

Documented engines are SGLang on NVIDIA B300 (CUDA 13.0) and AMD MI355X (ROCm 7.2), vLLM on NVIDIA B300, and experimental CUDA-only TokenSpeed. Kernels target NVIDIA sm_80 and newer and AMD CDNA2 to CDNA4 through Triton. Requests are text only, and pipeline parallelism is not supported. The README reports measured speedups over no-speculation controls, for example 1.91x on SGLang with 4x B300 at concurrency 8 and 2.99x on vLLM at concurrency 1.

Licensing Model

The license permits research, evaluation, education and noncommercial use. Commercial use is allowed within conditions including a US $20 million aggregate revenue threshold, and competing offerings need a separate written commercial license. Commercial inquiries go to licensing@eagle.ai.