TextGrad

TextGrad is an open-source Python package from the Zou Group at Stanford that implements automatic "differentiation" via text. It backpropagates natural-language feedback from large language models through a computation graph, using an API that mirrors PyTorch. The work was published in Nature in March 2025, and the latest listed release is v0.1.6.

What It Is

TextGrad is a framework for optimizing text-based variables with LLM feedback instead of numeric gradients. You define variables (such as an answer, a code snippet, or a system prompt), a natural-language loss function, and an optimizer. Calling backward produces textual gradients, which are interpretable feedback, and the optimizer step rewrites the variable accordingly.

How the Workflow Works

The core primitives follow PyTorch conventions: Variable, TextLoss, BlackboxLLM, and TGD (Textual Gradient Descent). A typical loop gets an initial LLM response, evaluates it with a text-specified loss, runs loss.backward(), and then calls optimizer.step() to refine the variable. The README demonstrates this on a reasoning question, a math solution, and prompt optimization, where a system prompt for gpt-3.5-turbo is improved using gpt-4o feedback.

Applications and Model Support

The project site describes applications in coding (LeetCode Hard problems), scientific question answering, reasoning prompt optimization, molecule design, and radiation treatment planning. Its authors report raising GPT-4o zero-shot accuracy on Google-Proof QA from 51% to 55%. An experimental engine built on litellm lets users connect any litellm-supported model, including Bedrock, Together, and Gemini, with optional caching. Multimodal optimization is covered in a tutorial.

Setup Path

Install with pip or conda-forge, with an optional vllm extra. Colab tutorials cover primitives, solution optimization, code loss definition, prompt optimization, and multimodal optimization. These require an OpenAI or Anthropic API key. The code is released under the MIT license.