Introducing CUDA Rust: Two Tracks for Writing GPU Kernels

In September 2026, NVIDIA announced it is leaning into native GPU programming in Rust. CUDA C++ and CUDA Python are mature, enterprise-grade toolchains, and NVIDIA will be growing and maturing CUDA Rust into 2027 and beyond The systems layer of AI spans inference engines, serving infrastructure, drivers, and agent runtimes, and it churns constantly as models and techniques change. More and more of it is written in Rust, which catches whole classes of bugs at compile time without giving up performance. NVIDIA is part of that shift for the same reason. The Nova Linux driver is written in Rust. NVIDIA Dynamo is built on a Rust core. NVTX has Rust bindings. The GPU kernel is the exception. You can launch kernels from Rust, but the kernel itself often has to be written in another language. NVIDIA CUDA Rust closes that gap. GPU kernels can be written in Rust, compiled natively to PTX, rather than a wrapper around code from somewhere else. There are two tracks to use Rust, matching the two tracks CUDA itself has. SIMT is the model you already write in CUDA C++ or numba-cuda. You indicate what one thread does, and launch thousands of them. Tile is a newer programming model, which is also available in C++ and Python. All of these frontends let you say what one tile of data does, and the Tile IR compiler does the rest. When you are picking one to build on, reach for Tile first. The compiler decides how tiles map onto each architecture, so your source doesn’t encode architecture-specific choices, and you drop to SIMT when you need that control or want to manage memory and threads yourself. Which language you reach for is a separate question from which model. Use the CUDA exposure that best fits the stack you already have. The two projects below are for when that stack is Rust. We plan to support inter-language interop, so the choice does not lock you out of the others. Below is the same kernel on each track, which performs elementwise addition over 1,024 floats. Both are complete programs, both run, and both print the same line, so you can read them side by side and see what changes. The SIMT track: cuda-oxide cuda-oxide is a custom rustc codegen backend. It intercepts compilation, routes #[kernel] functions through Rust MIR, the community Pliron IR framework, and LLVM IR down to PTX, and hands everything else to the standard backend. The GPU dialects on top of Pliron are ours. The dialects and every transform stay in Rust until the standard LLVM backend takes over. You will need Linux, a GPU with compute capability 8.0 or later, a CUDA toolkit (12.x or newer), clang with its libclang headers, and the pinned nightly toolchain. cargo oxide doctor checks all of it, including the optional system LLVM. Install cargo-oxide, the Cargo subcommand that drives the build: cargo +nightly-2026-04-03 install --git https://github.com/NVlabs/cuda-oxide.git cargo-oxide Then scaffold a project and run it. The template is a complete vector addition program: cargo oxide new vecadd_demo cd vecadd_demo cargo oxide doctor cargo oxide run The first cargo oxide run builds the codegen backend, so expect it to take a while. Later runs reuse the cache. It prints PASSED: all 1024 elements correct. This is the whole program that did it, exactly what cargo oxide new wrote, with comments added here: use cuda_device::{kernel, launch_bounds, launch_contract, thread, DisjointSlice}; use cuda_host::cuda_module; use cuda_core::{CudaContext, DeviceBuffer, LaunchConfig1D}; // === DEVICE CODE - everything in here is compiled to PTX === // The macro also generates the host-side API used further down: // `load`, `prepare_vecadd`, and the safe `vecadd` launch method. #[cuda_module] mod kernels { use super::*; #[kernel] // GPU entry point #[launch_bounds(256)] // max threads per block; lets the compiler budget registers #[launch_contract(domain = 1, block = (256, 1, 1))] // indexes in 1-D, 256-thread blocks pub fn vecadd(a: &[f32], b: &[f32], mut c: DisjointSlice<f32>) { let idx = thread::index_1d(); let idx_raw = idx.get(); // the plain usize, for reading the inputs if let Some(c_elem) = c.get_mut(idx) { *c_elem = a[idx_raw] + b[idx_raw]; } } } fn main() -> Result<(), Box<dyn std::error::Error>> { // === HOST SETUP - device, stream, and buffers === let ctx = CudaContext::new(0)?; let stream = ctx.default_stream(); const N: usize = 1024; let a_host: Vec<f32> = (0..N).map(|i| i as f32).collect(); let b_host: Vec<f32> = (0..N).map(|i| (i * 2) as f32).collect(); let a_dev = DeviceBuffer::from_host(&stream, &a_host)?; let b_dev = DeviceBuffer::from_host(&stream, &b_host)?; let mut c_dev = DeviceBuffer::<f32>::zeroed(&stream, N)?; // === LOAD, PREPARE, LAUNCH === // SAFETY: this package owns the embedded device bundle produced for the // kernels module above. let module = unsafe { kernels::load(&ctx)? }; // 4 blocks of 256 threads, 0 bytes of dynamic shared memory. `prepare_vecadd` // checks that against the contract above and against the live device limits. // The safe `vecadd` below takes that token where a raw config would go. let prepared = module.prepare_vecadd(LaunchConfig1D::new((N as u32).div_ceil(256), 256, 0))?; module.vecadd(&stream, &prepared, &a_dev, &b_dev, &mut c_dev)?; // === READ BACK AND VERIFY === // Copies down and synchronizes, so the launch has finished by the time // `c_host` can be read. let c_host = c_dev.to_host_vec(&stream)?; let errors = (0..N) .filter(|&i| (c_host[i] - (a_host[i] + b_host[i])).abs() > 1e-5) .count(); if errors == 0 { println!("PASSED: all {} elements correct", N); } else { eprintln!("FAILED: {} errors", errors); std::process::exit(1); } Ok(()) } Host and device code live in one file, build with one command, and need no separate kernel crate. Read the kernel signature first, because it carries the whole safety argument. a and b are ordinary shared slices, readable by every thread. c is a DisjointSlice<f32>, a type that hands each thread exclusive access to its own element and nothing else. It exists because &mut [f32] is the wrong shape for the job. Every thread would need the same &mut, which Rust correctly refuses. DisjointSlice splits that one mutable borrow into per-thread pieces. thread::index_1d() returns an index type, not a bare integer, and c.get_mut(idx) only accepts that type. You get back an Option, so the out-of-bounds case is a branch you handle rather than a memory error you find later. The launch is checked rather than trusted. #[launch_contract] declares that this kernel indexes in one dimension with 256-thread blocks. prepare_vecadd validates your LaunchConfig1D against that declaration and the live device limits, and hands back a proof that the safe vecadd method requires. Kernels without a contract expose only raw unsafe launch methods, because a bare LaunchConfig says nothing about the kernel it is launching. The Tile track: cutile-rs cutile-rs works one level higher. You perform computations on tiles rather than scalars. Each tile block runs the kernel body once as a single logical thread over one sub-tensor of data, and the compiler decides how many real GPU threads back it. The #[cutile::module] macro embeds the kernel’s AST in the host binary and JIT-compiles it through CUDA Tile IR (the NVIDIA tile-level compiler IR) when the kernel is first needed. Requirements are lighter than the SIMT track. You need a GPU with compute capability 8.0 or later, CUDA 13.3, stable Rust 1.89 or newer, and Linux, but no nightly toolchain and no LLVM of your own. cutile is published, so there is nothing to clone: cargo new vecadd_demo cd vecadd_demo cargo add cutile Here is the same elementwise addition, written for tiles. Paste it into src/main.rs and cargo run: use cutile::prelude::*; // The macro captures this module's AST into the host binary. The kernel is // JIT-compiled through CUDA Tile IR the first time it is actually launched. #[cutile::module] mod kernel { use cutile::core::*; #[cutile::entry()] fn add<const B: i32>( // B is the tile width, a static dimension. A different B produces a // different specialization. z: &mut Tensor<f32, { [B] }>, // exclusive output, one sub-tensor of B elements x: &Tensor<f32, { [-1] }>, // shared input; -1 is a dynamic dimension, resolved at launch y: &Tensor<f32, { [-1] }>, ) { // This body runs once per mut sub-tensor, as a single logical thread. // Tile kernels load tiles, not scalars, from x and y. let tx = load_tile_like(x, z); // the slice of x lining up with this sub-tensor of z let ty = load_tile_like(y, z); z.store(tx + ty); // elementwise across the whole tile } } fn main() -> Result<(), Error> { let device = Device::new(0)?; let stream = device.new_stream()?; // These are lazy. Nothing has touched the GPU yet. let x = api::ones::<f32>(&[1024]); let y = api::ones::<f32>(&[1024]); // Partitioning does three things at once: gives each tile exclusive // ownership of its own 128-element chunk, fixes the grid at 1024/128 = 8 // tiles, and supplies B. let z = api::zeros::<f32>(&[1024]).partition([128]); let c: Vec<f32> = kernel::add(z, x, y) // takes ownership of all three tensors .first() // ...and returns them; pick the output back out .unpartition() // drop the host-side partition wrapper; no data moves .to_host_vec() // record the copy back .sync_on(&stream)?; // and only now does any of it run let errors = c.iter().filter(|&&v| (v - 2.0).abs() > 1e-5).count(); if errors == 0 { println!("PASSED: all {} elements correct", c.len()); } else { eprintln!("FAILED: {errors} errors"); } Ok(()) } PASSED: all 1024 elements correct The Tile track reaches the same answer on stable Rust, and its signature makes the same safety argument. There is no DisjointSlice this time. Partitioning on the host is only needed for mutable tensors, and it hands each tile block one writable sub-tensor that no other tile block can overlap. That exclusivity is what &mut already guarantees.