rGPU

rGPU is an open-source tool, published under the Apache License 2.0, that runs GPU work on a remote NVIDIA machine while the application stays on the client. It offers a PyTorch device path and a CUDA shim path for existing Linux CUDA programs. The project documentation says a Mac with no CUDA installation can hold tensors and run PyTorch operations on a remote GPU.

What It Is

rGPU is a remote GPU access layer. Python code runs locally, and tensor operations and tensor storage live on a GPU server. Connections go through an SSH tunnel, and the rgpu-run launcher opens the tunnel and configures the connection.

Two Integration Paths

The PyTorch device path suits programs that can opt into an rgpu device (for example torch.randn(1024, 1024, device="rgpu")), and operations travel over TCP. The README describes this as the simpler integration. The CUDA shim path targets existing Linux CUDA programs, including stock CUDA PyTorch, through shims for libcuda, the CUDA Runtime, cuBLAS, cuBLASLt and cuDNN. The README notes that this path has a larger compatibility surface.

Setup and Security Notes

Install with pip install rgpu, deploy the server following the quickstart, then run a script with rgpu-run --host user@gpu-host python script.py. The documentation covers training with autograd, a nanoGPT example, configuration, performance measurement and troubleshooting. Neither protocol authenticates or encrypts connections itself, so the README advises keeping the operations server on its localhost bind and using SSH. It also says to restrict CUDA server port 9713 with firewall rules, since that server listens on all IPv4 interfaces. The site states it is built for trusted GPU hosts.