ONNX Runtime

ONNX Runtime is an open-source, cross-platform machine learning accelerator developed by Microsoft, licensed under the MIT License. It supports both inference and training workloads, enabling developers to run and optimize AI models across a wide range of hardware, operating systems, and programming languages. The project is hosted on GitHub under the microsoft/onnxruntime repository and has accumulated over 21,000 stars.

What It Is

ONNX Runtime (ORT) is a production-grade AI engine designed to accelerate the inference and training of machine learning models. It accepts models in the ONNX (Open Neural Network Exchange) format and supports conversion from popular frameworks including PyTorch, TensorFlow/Keras, scikit-learn, LightGBM, and XGBoost. The runtime applies graph optimizations and hardware-specific transforms to maximize performance, and it can leverage hardware accelerators—CPUs, GPUs, and NPUs—through a pluggable execution provider architecture.

Cross-Platform Language and OS Support

One of ONNX Runtime's defining characteristics is its breadth of platform and language support:

• Languages: Python, C#, C++, Java, JavaScript, Rust, and more
• Operating systems: Linux, Windows, macOS, iOS, Android
• Deployment targets: Cloud, edge, web browsers (via ONNX Runtime Web), and mobile devices (via ONNX Runtime Mobile)

This makes it practical for teams that need to deploy the same model across server, desktop, mobile, and browser environments without rewriting inference code.

Inference and Training Capabilities

ONNX Runtime covers two distinct workload types:

Inferencing: ORT powers AI in Microsoft products including Windows, Office, Azure Cognitive Services, and Bing, according to the project's own documentation. It optimizes for latency, throughput, memory utilization, and binary size, with additional model optimization techniques available for specific use cases.

Training: ONNX Runtime Training accelerates large model training on multi-node NVIDIA GPUs for transformer models, requiring only a one-line addition to existing PyTorch training scripts. It also supports on-device training, allowing developers to fine-tune inference models locally for personalized, privacy-respecting experiences.

Generative AI Support

The project includes onnxruntime-genai, a companion package that integrates generative AI and large language model (LLM) capabilities. According to the project site, it supports state-of-the-art models for image synthesis and text generation, including Hugging Face models such as Llama-2-7b and models from the Azure AI Machine Learning Studio catalog.

Update: ONNX Runtime v1.29.0

The latest release is v1.29.0, published on August 12, 2026, reflecting active and ongoing development. The repository was last pushed to on August 26, 2026, and the project maintains a public roadmap at onnxruntime.ai/roadmap. The GitHub repository tracks upcoming release dates, feature announcements, and guidance for feature requests, signaling a structured release cadence.

Open-Source Deployment Model

ONNX Runtime is released under the MIT License, making it free to use, modify, and distribute. It is installable via pip install onnxruntime or pip install onnxruntime-genai for the generative AI extension. Plugin execution provider repositories, such as the QNN Plugin EP for Qualcomm hardware, are maintained separately under the onnxruntime GitHub organization. Contributions are accepted via GitHub Issues and Discussions, and the project follows the Microsoft Open Source Code of Conduct.