LamBench

λ-bench (LamBench) is an open-source benchmark suite containing 120 pure lambda calculus programming problems designed to evaluate AI model capabilities in functional and symbolic reasoning. Each problem challenges a model to write a program in Lamb, a minimal lambda calculus language, using λ-encodings of data structures to implement specific algorithms. Models receive a problem description, data encoding specification, and test cases, then must return a single .lam program that passes all input/output pairs. The benchmark spans 12 categories ranging from trivial Church natural number arithmetic to highly complex tasks like BF interpreters, FFT, and Sudoku solvers — all in pure λ-calculus.

• 120 Diverse Problems — Problems are organized across 12 categories including Church Naturals, Scott Naturals, Church/Scott Lists, Trees, ADTs, N-Tuples, and complex Algorithms.
• Live Leaderboard — A generated GitHub Pages landing page displays up-to-date rankings for all evaluated models, built by running bun run build.
• Lamb Language — A minimal pure lambda calculus with named top-level definitions; no built-in data types — everything is λ-encoded using abstractions and applications.
• Automated Evaluation Harness — Run bun bench <provider/model> to evaluate any supported model; results are written as timestamped text files in the res/ directory.
• Flexible CLI Options — Supports --filter <prefix>, --concurrency <n>, --timeout <seconds>, and --no-reasoning flags for fine-grained benchmark control.
• Multi-Provider Support — Works with OpenAI, Anthropic, and Google model APIs; API keys are stored in ~/.config/ for easy configuration.
• v1 Scoring — Score is the pass rate (solved problems / 120); future versions will incorporate program size measured in bits against reference implementations.
• Reference Solutions Included — The lam/ directory contains reference .lam solutions for all 120 tasks, enabling size-based comparisons.