HybridAgent

HybridAgent is a research codebase from Together AI and collaborators at Stanford, UIUC and Meta that implements Hybrid Reasoning Agents. It provides a generation engine in which an LLM agent moves between latent reasoning and text interaction within a single generation, along with training stages, batch inference and an inspector for latent thoughts. Released checkpoints and datasets are hosted on Hugging Face.

What It Is

HybridAgent is an agent method and toolkit rather than a hosted application. A lightweight HybridLink maps last-layer hidden states back into the input-embedding space, so the agent can enter latent mode for inner thinking and return to text mode for tool calls, final answers and external monitoring. The hybrid context of text, latent thoughts and tool observations is carried across turns through the agent's working memory, and every latent thought can be rendered as readable pseudo-text.

Workflow and Outputs

The repository includes the hybrid generation engine with Python and search tools (concurrent tool calls are supported), HybridSFT and HybridRL post-training stages, and batch inference over a JSONL file of questions. Each batch record contains the hybrid trajectory, the extracted answer, the pseudo-text readout of every latent span and tool-call statistics. The Interpretable Latent Mode inspector renders latent thoughts inline with text reasoning, tool calls and observations as an HTML report or terminal view.

Models and Training Setup

The README lists HybridAgent-8B-Agentic and HybridAgent-14B-Agentic, built from Qwen3 base models through HybridSFT and HybridRL, plus HybridSFT and HybridRL datasets. Setup uses a conda environment with Python 3.11, PyTorch and a Tavily API key for the search tool. The README recommends 8 or 16 GPUs for HybridSFT and 32 GPUs (4 nodes of 8) for HybridRL.