# Jevlike

> An open-source Python library for training small models that select among a changing list of text options in a single forward pass, inspired by TypeSafe's Jev model.

Jevlike is an independent, MIT-licensed research starter for training small "option-selection" models that take a piece of text and a list of N text options and return one probability per option — all in a single forward pass rather than generating an answer token by token. It was created by Minimal Labs and published on GitHub, where it has accumulated 900 stars and 76 forks since its initial release. The project is explicitly inspired by TypeSafe's commercial Jev model but is an independent implementation; the README notes that TypeSafe has not published its design.

## What It Is

Jevlike trains a lightweight scorer that maps a context string and a variable-length list of text options to a probability distribution over those options. The core mechanism — an option-attention head — assigns attention weights from each option's query vector to the context tokens, produces one attended context vector per option, and converts the resulting scores into probabilities via softmax. This one-pass design is architecturally distinct from autoregressive decoders: the README reports that at eight options, one pass was approximately 100 times faster than a small decoder forced to write 400 tokens in local experiments.

## Architecture and Encoder Paths

The model supports two encoder strategies:

- **Default byte encoder**: Learns byte-level embeddings from scratch. Cheap and fast, but the README acknowledges it is "weak on language meaning." Truncates context to 192 bytes and each option to 32 bytes by default, with configurable limits.
- **Frozen pretrained encoder (optional)**: Integrates any compatible Hugging Face encoder (e.g., Qwen/Qwen2.5-0.5B) via an optional `[transformers]` install. The checkpoint stores only the trained scorer head and the encoder name — not the frozen encoder weights — so loading requires access to the same Hugging Face model.

The scorer head width is controlled by `--rank`, and training supports CPU, Apple MPS (Mac GPU), and CUDA (NVIDIA GPU).

## Included Examples and Checkpoints

The repository ships with two game-based demonstrations that use the same visual scorer from `jevlike.vision`:

- **Doom example**: Scores seven controller buttons from image patches. The released joint checkpoint averaged 0.60 kills and -97.50 reward across ten recorded episodes (per the README).
- **Chess example**: Scores five keys for a chess controller. The chess-only checkpoint scored 4 wins, 46 draws, and 0 losses in 50 sampled games against a random mover, but 0 wins, 2 draws, and 48 losses against Stockfish level 0 (per the README). The README explicitly notes these windows were selected for activity and are not typical-play or competence claims.

A Wikispeedia next-click example is also included, with a download script for the public SNAP archives.

## Performance Benchmarks from Local Experiments

The README documents results from the experiments that led to this starter:

- ~98% top-1 accuracy on synthetic menus with the byte encoder.
- 26% accuracy on target-disjoint Wikispeedia next-click data using a frozen Qwen2.5-0.5B encoder, versus ~8% for shuffled and random-encoder controls.
- 29% accuracy with a small model trained from scratch on 40,000 clicks.

The README explicitly states these numbers describe local experiments, not the quickstart run, and that the project does not claim equal quality with Jev or reproduce TypeSafe's private training method.

## Setup Path

The quickstart uses `uv` for environment management. After cloning, users run `uv venv`, install with `uv pip install -e '.[dev]'`, and then use four CLI commands: `jevlike-data` (generate synthetic data), `jevlike-train`, `jevlike-eval`, and `jevlike-predict`. Game extras and the Hugging Face encoder path are optional install extras.

## Features
- Single-pass option scoring (no token-by-token generation)
- Variable-length option lists per inference call
- Default byte-level encoder trained from scratch
- Optional frozen Hugging Face pretrained encoder integration
- CLI tools: jevlike-data, jevlike-train, jevlike-eval, jevlike-predict
- JSONL data format with per-row variable option counts
- Top-1 and top-3 accuracy evaluation with calibration error
- Shuffled-context control for sanity checking
- Visual scorer for image-patch option scoring (jevlike.vision)
- Doom and chess game controller examples with released checkpoints
- Wikispeedia next-click example with download script
- CPU, Apple MPS, and CUDA training support
- Configurable context and option token limits
- MIT License

## Integrations
Hugging Face Transformers (optional frozen encoder), Qwen2.5-0.5B (example encoder), ViZDoom (Doom game example), Chess engine (chess example), Playwright (film rendering), SNAP Wikispeedia dataset, Stockfish (chess evaluation)

## Platforms
WINDOWS, MACOS, API, DEVELOPER_SDK, CLI

## Pricing
Open Source

## Links
- Website: https://github.com/vinnylarouge/jevlike
- Repository: https://github.com/vinnylarouge/jevlike
- EveryDev.ai: https://www.everydev.ai/tools/jevlike