# Jeeves

> Open-source 9B reasoning decision model that answers yes/no, multiple-choice and rating questions through a Jev-compatible API.

Jeeves is an open-source project from PostHog that trains a Jev-style decision classifier on Qwen3.5-9B so it reasons before it decides. It accepts a state plus yes/no, multiple-choice and rating questions, and returns calibrated probabilities. The repository includes the training code, train/dev/test data builders, inference server, a Python SDK and a diffusion drafter for faster reasoning.

## What It Is

Jeeves is a decision model and serving stack. It uses Qwen3.5-9B with LoRA adapters and a pointer head that scores each answer option. The model produces a reasoning chain, then scores the options with a softmax whose temperature is fitted on a dev set. Requests use Jev's format: `noul` for yes/no, `choice` for multiple-choice and `score` for ratings, all in the same request.

## Training and Reported Results

Training has two stages: supervised fine-tuning, then CISPO reinforcement learning, followed by temperature calibration. The README reports 0.889 accuracy on its test data against 0.822 for Kev-9B and 0.857 for Jev. On the 231 public JevBench items it reports 0.935 against 0.866 for Jev. These are the authors' own figures. The README also notes that Jeeves trails Jev on knowledge-heavy questions such as MMLU and MMLU-Pro.

## Serving and Speed Options

The server is started with `python -m inference.serve` and runs on CUDA in bf16 or FP8. Inference on Apple Silicon (MPS) is also supported. A block-4 diffusion drafter speeds up reasoning, and the README reports about 1.6x on one question. Request options are:
- `think`
- `max_think`
- `nothink_threshold`
- `return_reasoning`

These trade latency against accuracy. The README reports about 0.3 s per request without thinking and a 3.3 s median with full thinking on one H100.

## Python SDK and Reproduction

The `sdk/` folder provides `jeeves_sdk`, a drop-in replacement for Jev's Python SDK. It connects to a local server by default and needs no API key. Scripts for building the data, training, evaluation, JevBench, export and FP8 quantization are included. Weights are published on Hugging Face.

## Limitations

The README states that thinking has a slow tail, with 17 s at p90 on one H100 with full chains. It also states that no language-consistency reward was used, so reasoning chains are not well interpretable.

## Features
- Yes/no, multiple-choice and rating questions in one request
- Reasoning before decision, trained with SFT and CISPO
- Jev-compatible HTTP API and drop-in Python SDK
- Block-4 diffusion drafter for faster reasoning
- FP8 and bf16 inference on CUDA and Apple Silicon (MPS)
- Adjustable thinking budget and no-think threshold
- Calibrated probabilities via temperature scaling
- Full training, data prep and evaluation code
- Export that fuses LoRA into a standalone model

## Integrations
Hugging Face, Qwen3.5-9B, Jev SDK (typesafe-sdk)

## Platforms
LINUX, MACOS, API, DEVELOPER_SDK, CLI

## Pricing
Open Source

## Links
- Website: https://github.com/PostHog/jeeves
- Documentation: https://github.com/PostHog/jeeves
- Repository: https://github.com/PostHog/jeeves
- EveryDev.ai: https://www.everydev.ai/tools/jeeves