Awesome Conformal Prediction

Awesome Conformal Prediction is a curated guide to conformal prediction (conformal inference), maintained on GitHub by Valery Manokhin, PhD. It collects tutorials, research papers, libraries in Python, R, Julia and other languages, and runnable examples covering prediction intervals, classification sets, time-series forecasting, anomaly detection, and LLM uncertainty.

What It Is

The repository is a reference list rather than a software package. It explains how conformal prediction turns model outputs into prediction intervals for regression or prediction sets for classification. In split conformal prediction, a model is fit on training data and a separate calibration set scores its errors to set a threshold. Under exchangeability, this gives finite-sample marginal coverage at the target level. The README notes that subgroup coverage and distribution shift need separate evaluation.

How the Collection Is Organized

A "Start here" table routes readers by goal: understanding the guarantee, building a first interval or set, choosing a package, or exploring applications. Sections cover recent research (coverage diagnostics, LLMs, vision, healthcare, science, robotics), libraries by language, AI platforms using conformal prediction, Kaggle notebooks, time-series forecasting, anomaly detection, tutorials, courses, books, articles, videos, slides, papers, theses, events, conferences and researchers. Entries marked with a fire icon are reviewed core references, and a clock icon marks time-series resources.

Hands-On Quickstart

A runnable notebook demonstrates regression and classification with NumPy and scikit-learn. The regression example uses synthetic data, fits a linear model, calibrates on held-out absolute residuals, and reports test coverage and mean interval width without any dataset download.

Author Materials

The README also promotes the author's books, Applied Conformal Prediction and Introduction to Conformal Prediction, and a paid live Applied Conformal Prediction course on Maven.