Ray

Ray is an open-source AI compute engine built around a distributed Python runtime and a set of high-level AI libraries. It is designed to scale machine learning and Python workloads from a single laptop to clusters of thousands of GPUs and CPUs, with native support for heterogeneous accelerators and fine-grained, independent scaling. Ray is Python-native and built by developers for developers, with simple primitives like @ray.remote decorators that turn ordinary Python functions and classes into distributed tasks and actors.

Ray originated at UC Berkeley's RISELab and is now stewarded by Anyscale alongside a large open-source community. According to the project site, Ray has 34.8k+ GitHub stars, 1,000+ contributors, and 40k+ repository downloads, and is used in production by companies including Instacart, Pinterest, Canva, and Amazon to handle workloads ranging from foundation model training to exabyte-scale data processing. The framework is released under the Apache License 2.0.

The project is split into Ray Core — the distributed runtime providing tasks, actors, and objects — and a set of AI Libraries that sit on top: Ray Data, Ray Train, Ray Tune, Ray Serve, and Ray RLlib. These cover multi-modal data processing, distributed training of traditional ML and Gen AI models, hyperparameter tuning, scalable model serving, and production-grade reinforcement learning. Ray also supports LLM inference, batch inference, and LLM fine-tuning workflows, and integrates with PyTorch, TensorFlow, JAX, XGBoost, and Kubernetes.

• Ray Core distributed runtime — Scale Python tasks, actors, and objects across CPUs, GPUs, and machines with a small set of primitives and decorators like @ray.remote.
• Ray Train — Distributed training for foundation models, time-series models, and traditional ML such as XGBoost, compatible with major ML frameworks.
• Ray Tune — Hyperparameter optimization with state-of-the-art search algorithms and distributed experiment execution.
• Ray Serve — Scalable model serving for ML models and LLMs, with independent scaling and fractional resource allocation.
• Ray Data — Distributed data processing for structured and unstructured data including images, video, and audio, used for ML pipelines and batch inference.
• Ray RLlib — Production-grade reinforcement learning with distributed RL workloads behind a unified API.
• LLM inference and fine-tuning — Online and batch LLM serving plus fine-tuning workflows that scale across heterogeneous accelerators.
• Cluster and cloud ready — Run on any cloud, Kubernetes, or on-prem, with autoscaling and integrations across the ML ecosystem.
• Apache 2.0 open source — Free to use, modify, and distribute under the Apache License 2.0; install with pip install ray.