# ort

> Open-source Rust binding for ONNX Runtime that runs hardware-accelerated inference and training on ONNX models.

ort is an open-source Rust interface from pykeio for running hardware-accelerated inference and training on machine learning models in the ONNX format. It is primarily a wrapper around Microsoft's ONNX Runtime and also supports other pure-Rust backends. The documentation describes version 2.0.0-rc.14 as production-ready.

## What It Is

ort is a Rust crate that lets developers deploy models exported from PyTorch, TensorFlow, Keras, scikit-learn, or PaddlePaddle by loading the resulting ONNX graph and running it through ONNX Runtime. It targets both on-device and datacenter deployment. The project is Apache-2.0 licensed.

## Workflow

Add the crate to Cargo.toml, convert your model to ONNX with the framework's exporter, then create an Environment and a Session. The session builder can set the graph optimization level, thread counts, and the model file. Prepare inputs, call run(), and extract output tensors as arrays.

## Performance and Backends

Execution providers enable hardware acceleration, including NVIDIA CUDA and TensorRT, Intel OpenVINO, Qualcomm QNN, and Huawei CANN. IoBinding controls where inputs and outputs are placed to improve I/O efficiency. Alternative backends include ort-candle, ort-tract, and ort-web for browser deployment.

## Ecosystem Use

The README lists open-source projects that use ort, including Text Embeddings Inference, Magika, FastEmbed-rs, and Ultralytics YOLO Rust Inference. The docs also cover platform support, Cargo features, linking, prebuilt binaries, and version mapping.

## Features
- Rust bindings for ONNX Runtime
- Hardware-accelerated inference and training for ONNX models
- Execution providers (CUDA, TensorRT, OpenVINO, QNN, CANN)
- IoBinding for I/O optimization
- Graph optimization levels and thread configuration
- Alternative backends: candle, tract, and web
- Prebuilt binaries and multiversion support

## Integrations
ONNX Runtime, PyTorch, TensorFlow, Keras, scikit-learn, PaddlePaddle, Hugging Face Optimum, NVIDIA CUDA, TensorRT, Intel OpenVINO, Qualcomm QNN, Huawei CANN, candle, tract

## Platforms
ANDROID, IOS, API, DEVELOPER_SDK, CLI

## Pricing
Open Source

## Version
2.0.0-rc.14

## Links
- Website: https://ort.pyke.io/
- Documentation: https://ort.pyke.io/
- Repository: https://github.com/pykeio/ort
- EveryDev.ai: https://www.everydev.ai/tools/ort