The end of responsive images
We believe that this document is fully human-written.
Hacker News Article AI Analysis
Content Label
Human
AI Generated
0%
Human
100%
Window 1 - Human
I’ve been waiting for fourteen years to write this article. Fourteen years to tell you about one relatively new addition to the way images work on the web. For you, just a handful of characters will mean improvements to the fundamental ergonomics of working with images. For users, it will mean invisible, seamless, and potentially massive improvements to front-end performance, forever stitched into the fabric of the web. For me, it means the time has finally come to confess to my sinister machinations — a confession almost a decade and a half in the making.Back then, I was the esteemed Chair of the RICG — the “pirate radio” web standards body responsible for bringing responsive image markup to the web platform. Some of you remember. Some of you were there at the advent of responsive web design, helping to find brand new use cases where the web platform fell short — as a scrappy band of front-end specialists rallied, organized, and crashed headlong into a web standards process that did not welcome them. We demanded a seat at the table alongside browser vendors, representing the needs of web designers and developers and the users we served. Our numbers swelled to the hundreds, and after years of iteration, countless scrapped draft specifications and prototypes, and endless arguments-turned-consensus across antique mailing lists and IRC channels, we finally arrived at a workable syntax hand-in-hand with browser vendors. Then we made it real — raised money from the community to fund independently-developed implementations in browsers, built the polyfills that would drive adoption, wired these new features up major CMSs, wrote articles and gave talks, and distributed — if I may say so — some of the best t-shirts the web standards game has ever seen.I imagine just as many of you weren’t there for any of that, as ancient as that history is in web development terms. For you, responsive image markup has been around as long as you’ve been making websites — a dense, opaque, inexorable, inescapable aspect of the web platform, an arcane syntax and a constant source of frustration.If you’re in the latter group, well, please allow me to introduce myself: I did that. Right here; eyes front — me.AdvertEvery time you tried and failed to figure out why the browser was selecting a certain source from srcset? You didn’t know it, but I was the one putting you through it.
Window 2 - Human
Every time you had to pull in some enormous third-party library to deal with a syntax very clearly not designed to be parsed by any human? Not only was I the cause, hell, I might have helped write it. When you ran some workflow-obliterating bookmarklet in hopes of generating a sizes value that mostly, kind of matched the reality of your layouts? When it was all too much; when you threw up your hands — gave up — and instead found yourself foisting huge source files upon countless users who might never see any practical benefit, but would bear all the performance costs? None of that was your fault. That was all me. Not only did I not stop these syntaxes from being standardized, I was the flag-bearer for responsive images — I fought tooth-and-nail for the markup you’ve cursed.Oh-ho, and as if that wasn’t enough, here’s the part that will really make you mad: I hate it all too.Every talk I gave and article I wrote on the subject — the course I wrote about images, the entire book I wrote about images — all done through gritted teeth. There are parts of this syntax that I’ve hated since the moment I first set eyes on them — which, again, was the very same moment that I became their most vocal champion. I’m not sorry. I’d do it again.The BeastDon’t get me wrong: I don’t hate responsive images. The problem needed solving, there are no two ways about that. Then, as now, the vast majority of a website’s transfer size is in images. A flexible image requires an image source large enough to cover the largest size it will occupy in a layout — without responsive images, an image designed to occupy a space in a layout that’s, say, two thousand pixels wide at its largest layout sizes would mean serving every user an image source at least two thousand pixels wide. Scaling that image down to suit a smaller display is trivial in CSS, but the request remains the same — the user bears all the transfer costs, but sees no benefit from an enormous image source.Remember, too, that this problem stems from an era where sub-3G connections were still common.
Window 3 - Human
There was no reliable way to tailor those requests to a user’s browsing context in a way that maintained browser-level performance optimizations — and ultimately, the solutions we got were effective, performant, and have saved unfathomable amounts of bandwidth for users. Responsive images, as a concept, are an incredible addition to the web platform. I’m proud to have been able to play a small part in it.Hell, it’s not even that I wholesale don’t like the responsive image syntaxes. Not all of them, anyway. picture I liked from the very beginning. Granted, that’s a prescriptive syntax, and it represents a very different set of use cases from “I just want fast images.” The picture element is for control — the siren song that has called out to designers and developers of all stripes since time immemorial, and I’m no exception. Control over sources, control over the conditions used to determine whether they’re requested, even control over whether the browser should bail out of the source selection algorithm entirely to the tune of “nevermind, don’t load any source” — it took me a while to come around on that last one, but I got there.What’s not to like? Who wouldn’t want that level of fine-grained control? Not only that, but picture made it possible to responsibly serve brand new image formats with fast, reliable fallbacks across browsers, opening the door for incredible advances in encoding and compression without the need for a single scrap of JavaScript. The syntax makes perfect, readable sense, it provides us with a template for standardizing smarter decisions around all media requests, and it grows ever more powerful as more and more media queries are added to the platform. picture is great. I like picture; everyone likes picture. We’re not here to talk about picture.Advertpicture is something altogether different from srcset and sizes, which represent a descriptive syntax. You use srcset to provide the browser with information about a set of image sources, identical apart from their dimensions, and sizes to provide the browser with information about how the image will be rendered, and at no point do you use either to tell the browser what to do with any of it. Once given this information, the browser can then use it to do exactly one (1) very complicated thing: determine the image source most appropriate for that user’s browsing context.
Window 4 - Human
Visually, the source selected from the list of candidates in srcset doesn’t matter to the user — the sources will all look the same — but the chosen candidate will best fit the user’s browsing context. You don’t get any control over how that decision is made. In fact, you don’t even get to know how that decision is made, by design — right down to an “explicitly vague” step in the source selection algorithm, carved into the HTML specification itself:
In an implementation-defined manner, choose one image source from sourceSet.
— Source
If something is said to be implementation-defined, the particulars of what is said to be implementation-defined are up to the implementation. In the absence of such language, the reverse holds: implementations have to follow the rules laid out in documents using this standard.
— Source
Unsettling, isn’t it? “Then the browser,” in strict technical terms, “just does whatever.” That formally codified lack of control didn’t just happen; that buck could have stopped with me, but no. Instead, I personally thumbs-upped the decision that you should not have any say in how srcset/sizes work — that you can’t even know how they work. Now, after all these years — with this, the reveal that I’ve been the villain of the story all along — I can finally tell you why. You’re not gonna like it one bit, either. It’s because I know you would have done it wrong.A human workDon’t take it too personally, I would’ve done it wrong too. Hell, I did do it wrong, through countless proposals and prototypes, in search of a solution that could be standardized — everybody did. In the end, all that iteration only proved that nobody could have gotten this part right. That “one thing” that srcset/sizes does — determining the image source best tailored to a user’s browsing context, including viewport size, display density, user preferences, bandwidth, and countless other potentially unknowable factors? Those factors include things we can’t know, and just as many things we shouldn’t know.For example, we can’t tailor asset delivery to a user’s connection speed, which seems like a shame. For a moment, though, let’s imagine we could — imagine we were able to say “use that source above this speed, and that source below it.”
Window 5 - Human
Now that those decisions are yours to control: what connection speed thresholds would you set for your image sources, and what would I set for mine? They’re different, I bet. That means that for a given connection speed, a user might get beautiful but bandwidth-obliterating image sources on one site, and highly compressed but wonderfully efficient ones on the next one. Which of those does that user actually want? Well, trick question, they’d all want something different, wouldn’t they? What would your organization want? Uh oh. Everyone is looking to you now — you, with the open tickets, and a meeting in half an hour, and all this control foisted upon you by the specification. Why does the website feel so slow? Why do our images look worse than our competitors’ now? Why does the website feel so slow again? Even when we’re only considering connection speed, the cost of our having more control is the user giving up theirs, and that’s before we’ve considered every other factor besides connection speed.I didn’t want that; I didn’t want that for the people who build the web, I didn’t want that for people using the web, and I sure as hell didn’t want to see the web itself buckle under the strain of a million massive image files backed by a hundred thousand figure out our responsive images policy in excruciating detail when we have time issues buried in trackers forever.AdvertThe browser has access to a lot more information than we do — certainly more than we should reasonably want access to — so it can make decisions about screen size and display density and bandwidth and user preferences and any number of future factors we can’t even imagine, without making any of it our problem. The browser can decide how to finesse details, like avoiding wasted requests by retaining larger sources rather than requesting functionally identical smaller ones if the larger sources already exist in the cache — I wouldn’t want to own that logic. The browser can poll preferences set by a user, to give them control over these decisions and ensure a consistent experience from one site to the next.Ultimately, we don’t need control when it comes to optimizing an image request. We just want faster images, and srcset and sizes cover that use case handily — better than you or I ever could, if we had to. It would be miserable if we had to.