# rGPU

> Runs PyTorch and CUDA workloads on a remote NVIDIA GPU while the application code stays on the local machine.

rGPU is an open-source tool, published under the Apache License 2.0, that runs GPU work on a remote NVIDIA machine while the application stays on the client. It offers a PyTorch device path and a CUDA shim path for existing Linux CUDA programs. The project documentation says a Mac with no CUDA installation can hold tensors and run PyTorch operations on a remote GPU.

## What It Is

rGPU is a remote GPU access layer. Python code runs locally, and tensor operations and tensor storage live on a GPU server. Connections go through an SSH tunnel, and the `rgpu-run` launcher opens the tunnel and configures the connection.

## Two Integration Paths

The PyTorch device path suits programs that can opt into an `rgpu` device (for example `torch.randn(1024, 1024, device="rgpu")`), and operations travel over TCP. The README describes this as the simpler integration. The CUDA shim path targets existing Linux CUDA programs, including stock CUDA PyTorch, through shims for libcuda, the CUDA Runtime, cuBLAS, cuBLASLt and cuDNN. The README notes that this path has a larger compatibility surface.

## Setup and Security Notes

Install with `pip install rgpu`, deploy the server following the quickstart, then run a script with `rgpu-run --host user@gpu-host python script.py`. The documentation covers training with autograd, a nanoGPT example, configuration, performance measurement and troubleshooting. Neither protocol authenticates or encrypts connections itself, so the README advises keeping the operations server on its localhost bind and using SSH. It also says to restrict CUDA server port 9713 with firewall rules, since that server listens on all IPv4 interfaces. The site states it is built for trusted GPU hosts.

## Features
- Remote PyTorch device (device="rgpu") with tensors held on the remote GPU
- Autograd and model training on a remote GPU
- CUDA shim for existing Linux CUDA programs
- Shims for libcuda, CUDA Runtime, cuBLAS, cuBLASLt and cuDNN
- rgpu-run launcher that opens an SSH tunnel
- Runs from a Mac with no CUDA installation
- Performance measurement guidance for host waits and transfers

## Integrations
PyTorch, CUDA, cuBLAS, cuBLASLt, cuDNN, SSH

## Platforms
MACOS, LINUX, API, DEVELOPER_SDK, CLI

## Pricing
Open Source

## Links
- Website: https://rgpu.dev
- Documentation: https://rgpu.dev/docs/
- Repository: https://github.com/ymcrcat/rgpu
- EveryDev.ai: https://www.everydev.ai/tools/rgpu