Computing Camera Rays

Published 2026-06-20

We are in a transition period, where ray tracing and rasterization coexist. Typical real-time rendering pipelines still use rasterization (often with deferred shading) for primary visibility and then trace rays for shadows, reflections or global illumination. Though, we are getting to the point where one can seriously consider to ditch rasterization and to use ray tracing for primary visibility as well, at least on some platforms. Then you need to compute camera rays, characterized by ray origin, ray direction and ray length, in a way that is consistent with what you would otherwise do for rasterization. In rasterization, you commonly use a world to clip space transformation matrix, also known as view-projection matrix to specify the camera. In this blog post, I will derive how to compute camera rays based on such a matrix.

The goal is to get something that works equally well for perspective and orthographic projection and whatever else such a matrix may represent. The obvious approach turns out to be prone to numerical cancellation and I present an alternative that works much more reliably. Overall, this is not a hard problem: For any given camera model (e.g. perspective projection with known field of view), it is quite easy to come up with an ad hoc solution that will work. Though, I find it valuable to have a solution based on readily available transformation matrices that does not require any further tinkering per camera model. If you do not care about the derivation, feel free to just copy the shader code in Listing 4 (which I hereby release into the public domain, like all code in this blog post).

Camera rays in clip space

The rasterization pipeline and clip space rely heavily on homogeneous coordinates, so let us begin by reviewing that. If we have a point with 3D Cartesian coordinates \((x^\prime, y^\prime, z^\prime)^\mathsf{T}\), we can get homogeneous coordinates for that point by simply attaching a 1 as fourth coordinate: \((x^\prime,y^\prime,z^\prime,1)^\mathsf{T}\).

This still describes a 3D point, but we have now gained the freedom to scale its coordinates by any non-zero factor \(w\neq 0\), which gives us

\[(x,y,z,w)^\mathsf{T} = (wx^\prime, wy^\prime, wz^\prime, w)^\mathsf{T}\text{.}\]

No matter how we choose \(w\), these coordinates still describe the same point. We can recover the inhomogeneous coordinates (i.e. dehomogenize) by dividing by the fourth component \(w\):

\[\frac{1}w(x,y,z,w)^\mathsf{T} = \left(\frac{x}{w}, \frac{y}{w}, \frac{z}{w}, 1\right)^\mathsf{T} = (x^\prime,y^\prime,z^\prime,1)^\mathsf{T}\text{.}\]

Homogeneous coordinates make many formulas simpler. For example, we will see below that we can also write down homogeneous coordinates for planes in 3D space, and then to check whether a point is on a plane, we just take a dot product. They also allow us to express translation with \(4\times 4\) matrices. Furthermore, they are useful for rasterization with a perspective projection. A perspective projection inherently requires us to perform a division at some point. With homogeneous coordinates, this division happens shortly before rasterization and it is simply the dehomogenization mentioned above.

Along with this design of rasterizers comes the notion of clip space. For screen space, we use a coordinate frame where coordinates \(x^\prime_c\) and \(y^\prime_c\) range from -1 to 1 across the extent of the camera frustum (the subscript \(c\) stands for clip space). In homogeneous coordinates, these bounds translate to \(-w_c\leq x_c\leq w_c\) and \(-w_c\leq y_c\leq w_c\). In addition, we define a near and far clipping plane based on the clip space z-coordinate. The far clipping plane is at \(z^\prime_c=1\), which translates to \(z_c\leq w_c\). The near clipping plane is defined differently, depending on the API: For Direct3D, the inequality is \(0\leq z_c\).

For OpenGL, the default behavior is that the near clipping plane is at \(-w_c\leq z_c\), but this has been made configurable through the extension GL_ARB_clip_control, which lets you choose the Direct3D behavior of \(0\leq z_c\). This extension has moved into core functionality in OpenGl 4.5. For Vulkan, the behavior is similarly configurable. To account for these differences, I will use a variable \(z^\prime_n\) which is \(0\) for the Direct3D conventions and \(-1\) for the old OpenGL default, i.e. either way the near clipping plane is \(z^\prime_n w_c\leq z_c\).

A typical renderer will use lots of other coordinate frames, such as camera space and object space for every single object, but the only other one that we care about here is world space, because we want to get the ray in world space. In the context of ray tracing, it is pretty easy to define what we mean by world space: It is whatever the top-level acceleration structure uses. For rasterization, we then prepare the world to clip space matrix \(M_{w,c}\in\mathbb{R}^{4\times4}\) that transforms points from world space to clip space. If \(p_w=(x_w, y_w, z_w, w_w)^\mathsf{T}\) denotes the world-space coordinates of a point, the corresponding clip-space coordinates are

\[p_c := (x_c, y_c, z_c, w_c)^\mathsf{T} := M_{w,c} p_w\text{.}\]

I am using OpenGL conventions here, which are consistent with common linear algebra conventions. In Direct3D conventions, the matrices and vectors would be transposed and the order of multiplication would be reverted (not sure who thought that was a good idea or why). More on that below.

Now if we have screen-space coordinates \(x^\prime_c, y^\prime_c\) ranging from \(-1\) to \(1\), we can easily determine points on the near and far clipping planes for that pixel in terms of their homogeneous coordinates in clip space.

They are

\[n_c := (x^\prime_c, y^\prime_c, z^\prime_n, 1)^\mathsf{T} \quad\text{and}\quad f_c := (x^\prime_c, y^\prime_c, 1, 1)^\mathsf{T}\text{,}\]

respectively. Our goal is to turn that into a ray origin and ray direction in world space. Sounds simple enough, right?

World space ray origin

As far as the ray origin is concerned, it really is simple. We take the point on the near clipping plane, transform it to world space and dehomogenize. To do so, we need the clip to world space transformation matrix \(M_{c,w}:=M^{-1}_{w,c}\). Then the point we want is simply \(n_w := M_{c,w} n_c\). Since ray tracing APIs do not deal with homogeneous coordinates, we then want to dehomogenize that, i.e. divide by the w-coordinate. Listing 1 provides an implementation in GLSL.

Listing 1: Returns a point on the near clipping plane for a camera with the given clip to world space transform. When clip_space_xy is \((-1, -1)\) the point is at the left top of the viewport, for \((1, -1)\) it is at the right top and for \((1, 1)\) it is at the right bottom (at least with default OpenGL, Vulkan and Direct3D behavior).

vec3 get_camera_ray_origin( vec2 clip_space_xy, mat4 clip_to_world) { // With old OpenGL conventions, use z_n=-1 float z_n = 0.0; vec4 n_c = vec4(clip_space_xy, z_n, 1.0); vec4 n_w = clip_to_world * n_c; return n_w.xyz / n_w.w; }

Bad approach for the ray direction

To describe the camera ray, we need the ray direction in addition to the ray origin. The obvious strategy is as follows: We use the code snippet above with z_n=1 to compute a point on the far plane. Then we take the difference of the points on the far and near plane and normalize this vector. From a mathematical point of view, there is nothing wrong with this approach.

It gives the correct result and it is also quite efficient to compute. Nonetheless, I strongly advise against using it. In fact, I will not even provide a code listing for it, lest somebody copies it by accident.

Instead, let us just look at the results of this approach in Figure 1. I like a good graphics glitch as much as anyone, but still this is not quite the result we were going for. A slow, continuous camera motion turns into a jittery mess. What is going on?

Figure 1: A video with a camera that moves slowly along a straight line. It is quite far from the origin and looks at a scene centered around the origin (namely the Lumberyard Bistro). Due to rounding errors in the computation of ray direction vectors, the camera motion appears jittery.

Looking at the last two columns of the clip to world space matrix \(M_{c,w}\), we see something like this:

-5995.15332031 5994.86083984 11787.78613281 -11786.94140625 -4032.25268555 4031.86547852 -50.01052094 50.01062393

Note how these two columns are almost identical, except for their sign. When we compute points on the far plane, we compute \(M_{c,w}f_c\) and the last two entries of \(f_c\) are 1. Thus, as we compute this matrix-vector product, we are summing the last two columns of the clip to world space matrix. Since they are nearly equal, but with opposite sign, this addition causes a catastrophic cancellation: These large floats have large absolute rounding errors and their tiny difference will have a similarly large absolute rounding error. Then the point on the far plane has reduced precision. The exact way in which this rounding error works out depends on the exact entries, which in turn depend on the camera position. Hence the unsmooth result in Figure 1.

Computing the clip to world space matrix in double precision arithmetic (before casting it to float when it is passed to the shader) diminishes the magnitude of these artifacts, but does not eliminate them. There may be other issues at work here, but this cancellation seems to be the main culprit.

Good approach for the ray direction

Let us try something different then. Instead of characterizing the line for the camera ray by specifying two points \(n_c, f_c\) on it, we will define two planes such that the line is the intersection of these two planes. We characterize these two planes using homogeneous coordinates

\[G_c:=(1, 0, 0, -x^\prime_c)^\mathsf{T} \quad\text{and}\quad H_c:=(0, 1, 0, -y^\prime_c)\text{.}\]

By definition, a point in clip space \(p_c\in\mathbb{R}^4\) is on the plane \(G_c\) if and only if \(G_c^\mathsf{T} p_c = 0\). In this equation, we are multiplying a row vector (due to the transpose) by a column vector, which gives us a dot product. That expands to

\[G_c^\mathsf{T} p_c = x_c - x^\prime_c w_c = 0\text{,}\]

which is equivalent to

\[\frac{x_c}{w_c} = x^\prime_c\text{.}\]

Thus, this plane encompasses all points that have the inhomogeneous clip-space x-coordinate \(x^\prime_c\). In particular, it contains all points on our camera ray. In a similar fashion, we can derive that

\[H_c^\mathsf{T} p_c = 0 \quad\Leftrightarrow\quad \frac{y_c}{w_c}=y^\prime_c \text{.}\]

So all points \(p_c\) on the camera ray are in the intersection of the two planes, i.e. they satisfy

\[G_c^\mathsf{T}p_c = H_c^\mathsf{T}p_c = 0\text{.}\]

The next step in the derivation is to transform these planes into world space.