gpt-go

gpt-go is an open-source educational implementation of a GPT-style transformer written in pure Go by zakirullin. It trains a small language model on Jules Verne books and can generate text in that style. The repository is written to be readable, and the README presents it as a companion to Andrej Karpathy's Neural Networks: Zero to Hero course.

What It Is

gpt-go is a from-scratch GPT implementation optimized for understanding rather than efficiency. Running go run . trains the model on the bundled text corpus, and the weights are saved to a model-1.234M file. Rerunning picks up the saved weights and continues training. The README says training takes about 40 minutes on a MacBook Air M3. Running go run . -chat skips training and goes straight to chat-only mode. Users can train on their own text by pointing the data.dataset variable at another corpus.

How to Learn From It

The repository is meant to be studied alongside the Zero to Hero course. Git tags show how the model evolved: naive, bigram, multihead, block, residual and full. The main_test.go file explains concepts step by step, starting from a single neuron example and building up to the self-attention mechanism using a triangular matrix.

Design Choices

The project deliberately drops the batch dimension so the math stays in 2D matrices, which are easier to build intuition with. Gradient accumulation was tried and removed because its effect was negligible. The gonum dependency was also removed. It gave about a 30% matmul speedup, but the author preferred a 40-line plain-Go matmul implementation. The README credits the itsubaki autograd package.