What Would It Mean to See a New Color?

During his first year as a professor of computer science at the University of California, Berkeley, Ren Ng was hurriedly putting together a survey course on computer graphics. In the syllabus he had inherited, a full week had been devoted to the subject of color. Ng thought that was a bit much. “I’m, like, Come on. It’s R.G.B.,” he said, referring to the red, green, and blue subpixels that constitute anything you see on a screen—your cluttered desktop, a Sahara-desert screen saver, the stream of the Netherlands-Japan World Cup game. Ng started gathering slides that would cover the wavelengths of light, the biology of the human eye—the basics—“Blah, blah, blah,” he said. A colleague shared a slide that he thought might be useful. It included a minutely detailed photograph of a patch of retina, seen through a microscope, which was attributed to Austin Roorda, a professor of vision science and optometry just across campus. Roorda’s lab had helped develop technology that could map the layout of individual cone cells—those primarily responsible for perceiving color—and that, furthermore, could target a single cone cell with light. Eyes are constantly moving; cone cells are extremely small; how color is translated from the millions of cone cells to the mind remains pretty mysterious; this was awesome work. Roorda’s lab was using the new technology to explore eye disease and the mechanics of how we see. Ng had his own notion, though: he wondered if it could be used to see a color that had never been seen before.To understand what Ng had in mind requires knowing a bit of the blah, blah, blah of color vision. We humans experience three primary colors not because the world is fundamentally composed of three colors but because our retinas typically have three kinds of color-perceiving cone cells. L cone cells respond to the relatively longer wavelengths of visible light, M cone cells to the medium wavelengths, and S cone cells to the shorter ones. In effect, this means that L cells respond most strongly to red light, M to green, and S to blue. But when you look at your hand—or a blade of grass, or a clear blue sky, or a fire truck—it is always some mixture of L, M, and S cone cells that are being stimulated.Ng’s idea was to use the Roorda lab’s technology to stimulate an array of cone cells in a manner that would never occur naturally. Ng said, “I e-mailed him, basically, What would happen if you stimulated only the M cells? Would that be like the greenest green, or what?” Roorda did not reply. This was 2016. Ng taught his computer-graphics course, pursued other research, and mostly forgot about his query. A year later, when he taught the course a second time, his curiosity returned, so he reached out to Roorda again. No response. When Ng was teaching the course for a third time, in 2018, he realized that he really was very curious about this question. He composed a lengthy note to Roorda, organized like a research proposal, titled “The Grass Is Greenest in Oz Vision.” In it, Ng imagined a future where people wore “Oz Vision eyeglass displays,” which would be based on the retinal cone-cell mapping and laser stimulation that Roorda’s lab was already doing. The Oz glasses would activate any pattern of retinal cone cells one chose. Ng’s hypothesis was that, if retinal cells were stimulated in ways that don’t occur naturally, “the set of perceivable colors” would be “significantly larger than the natural gamut of the human eye.” He imagined people discussing “the indescribable green of the grass in Oz,” then concluded that, although the Oz display “is science fiction,” his note was “a real proposal for joint research.” Roorda finally replied, proposing coffee. “I get a lot of e-mails suggesting what I should research,” Roorda told me, of the time it took him to respond.In the children’s novel “The Midnight Fox,” by Betsy Byars, the main character daydreams about digging in his back yard and coming across a “brand-new color.” A friend of mine remembers being captivated by this scene, and trying to picture the color. “I felt like I could conceive of it, but I couldn’t see it,” she said. The eighteenth-century Scottish philosopher David Hume considered at length whether a person who had seen every shade of blue except for one would be able to picture that one un-experienced shade; he concluded that the answer was yes. In my youth, I spent an afternoon wondering if I would have been able to imagine the fluorescent yellow of highlighter markers if I’d never seen it.When I first read about Ng and Roorda’s work, I tried to visualize what it would mean for there to be a new color. Where would it go in a color wheel? If you think of color as a property of a specific wavelength of light—which is how I thought of it—then you run into the impossibility of there being a “new” visible wavelength. As humans, we can see wavelengths from roughly 380 nanometres (which looks violet to us) to about 750 nm. (which looks red). Wavelengths shorter than 380 nm., which we’d call ultraviolet, are invisible to us (but not to bees or hummingbirds), as are wavelengths longer than 750 nm., which we refer to as infrared (and which snakes and salmon can perceive). The “greenest green” that Ng had in mind was neither ultraviolet nor infrared. It was not a new wavelength at all. If I wanted to picture this green, or at least try to, I would first have to understand that even the old familiar colors are much more complex than a particular wavelength of light.I went to visit Julian Kreimer, an artist and art professor at SUNY Purchase who has taught classes on color for the past twenty years. When we were nineteen years old, we took the same drawing course. The teacher, whom we revered, had us work for weeks in black. Inky blacks, charcoal blacks, waxy blacks—but only black. Kreimer had since gone on to other hues. The molding on the front door of his house, overlooking Green-Wood Cemetery, in Brooklyn, is painted in a series of colors, beginning with stripes of bubblegum pink, followed by orangeade, dark navy, vanilla cream, dark chocolate, lime, and then pink again. The effect is candy-shop colorful. The door’s background hue, which Kreimer said was battleship gray, looked violet to me. When he painted the door, he was “thinking about how neutrals make a small amount of saturated color appear more colorful.” He added, “I wanted to judo color—get the maximum chromatic effect with the least color.”I followed Kreimer up narrow stairs to his studio, where he walked me through some of his favorite books and lectures, focussing on those that would illuminate the idea of a green of unprecedented greenness. “There’s hue, value, and saturation,” he said—the three dimensions of color as they are generally thought of today. Hue is color, as in orange, yellow, red, etc. Value is how much white or black is present. Saturation is often described as brightness. The first color wheel, developed by Isaac Newton, in 1666, is a circle comprising seven wedges, each a hue of the rainbow as he (somewhat mystically) divided it: red, orange, yellow, green, blue, indigo, and violet. There is no pink, because pink is a red of another value. The German Romantic painter Philipp Otto Runge captured an additional dimension of color in his color sphere, which places hues at the equator, with the varying values of those hues becoming lighter or darker in respect to white at one pole and black at the other. The third dimension of color, saturation, is the variable that Ng and Roorda would play with in pursuit of a green of supernatural intensity. Intuitively, we tend to experience saturation as the purity of a color.Kreimer showed me a painted twelve-square-by-twelve-square grid from the 1961 book “The Art of Color,” by Johannes Itten. Kreimer described Itten as a “monkish guy who would shave his head and wear robes at the Bauhaus.” Itten’s grid proceeds from dark to light vertically, and from violet to red horizontally. Kreimer asked me to point out the colors that appeared most pure. The yellowest yellow was closest to the light end of the grid, and the orangiest orange and the tealest teal were nearer the middle. The bluest blue and the purplest purple were nearest to the grid’s dark base. Tracing a line through the “purest” colors forms a skewed bell curve. “So it’s weird, right?,” Kreimer said. “Why is pure yellow bright and pure blue dark?” Itten was a painter, but, as with many artists, his work on color constituted a scientific inquiry. The curve formed by the distribution of the most saturated colors hints at several characteristics of the human eye. (A bee would disagree on which squares were the purest, as would a hummingbird, a snake, or a salmon.) Because we have many more cones sensitive to the middle wavelengths of a rainbow—which centers on yellow—these wavelengths appear the brightest to our eyes.“Did she have a parrot? Cases are much easier to solve when there’s a parrot.”Cartoon by Sophie Lucido Johnson and Sammi SkolmoskiRoorda was the first guinea pig in the Oz research project. In a dark lab, he sat in a chair, eyes dilated, with his head position stabilized by a bite bar. His wife and his daughter had sometimes teased him, saying that, though he had spent years studying color vision, he still didn’t notice much of a difference between purple and magenta. “I think they just grew up paying a little more attention than I did,” he said. So perhaps he would not be a great test subject. Or maybe there would be a technical failure. Or the experiment would succeed in exclusively stimulating M cones, but the brain would not know what to do with input that nothing in evolution had primed it to interpret.Hannah Doyle, a graduate student who had been key in developing the experimental protocol and the software, was operating what they called the Oz Vision system, monitoring the optical correction and making sure Roorda maintained alignment. “We were using 543-nanometre light,