# gpt-go

> A tiny GPT language model implemented from scratch in pure Go, designed for learning and trained on Jules Verne books.

gpt-go is an open-source educational implementation of a GPT-style transformer written in pure Go by zakirullin. It trains a small language model on Jules Verne books and can generate text in that style. The repository is written to be readable, and the README presents it as a companion to Andrej Karpathy's Neural Networks: Zero to Hero course.

## What It Is

gpt-go is a from-scratch GPT implementation optimized for understanding rather than efficiency. Running `go run .` trains the model on the bundled text corpus, and the weights are saved to a `model-1.234M` file. Rerunning picks up the saved weights and continues training. The README says training takes about 40 minutes on a MacBook Air M3. Running `go run . -chat` skips training and goes straight to chat-only mode. Users can train on their own text by pointing the `data.dataset` variable at another corpus.

## How to Learn From It

The repository is meant to be studied alongside the Zero to Hero course. Git tags show how the model evolved: naive, bigram, multihead, block, residual and full. The `main_test.go` file explains concepts step by step, starting from a single neuron example and building up to the self-attention mechanism using a triangular matrix.

## Design Choices

The project deliberately drops the batch dimension so the math stays in 2D matrices, which are easier to build intuition with. Gradient accumulation was tried and removed because its effect was negligible. The gonum dependency was also removed. It gave about a 30% matmul speedup, but the author preferred a 40-line plain-Go matmul implementation. The README credits the itsubaki autograd package.

## Features
- GPT transformer implemented from scratch in pure Go
- Trains on Jules Verne books by default
- Custom dataset support via data.dataset variable
- Saves weights and resumes training on rerun
- Chat-only mode with -chat flag
- Git tags showing step-by-step model evolution
- Test file with explanations from a single neuron to self-attention
- No batch dimension, for simplicity

## Platforms
API, CLI

## Pricing
Open Source

## Links
- Website: https://github.com/zakirullin/gpt-go
- Repository: https://github.com/zakirullin/gpt-go
- EveryDev.ai: https://www.everydev.ai/tools/gpt-go