How Video Games Inspire Great UX – Scott Jenson

The Wayback Machine - http://web.archive.org/web/20260922124200/https://jenson.org/games/“Perspective is worth 80 IQ points.”Alan KayThis quote is meaningful to me as I’ve often struggled with a problem until an insight from a user test “changed my perspective”.I had this intuition that video games would also change my perspective. They seemed so much edgier, playing with deeper and more experimental UX techniques. I wanted to go well beyond classic “gamification”. That was all the rage a decade ago where apps used toys like “achievements” to encourage app use. Of course, given what we’ve learned in the last few years about the overuse of mobile phones and social media, it also seems irresponsible to make our apps “sticky” or “addictive”. I was looking for something deeper. Games felt more visceral and I wanted to understand them better and see where it took me.I’m an avid gamer but they felt a bit foreign to me as a UX designer. For example, one of the first game terms I learned was “juicy”, which refers to the amount of video effect you get for any action. Juicy games feel just a bit overwhelming but in a good way. This might be fun but likely counter productive. I wasn’t looking to overwhelm the user.My other concern was that games create tension. When you approach a fight in a game, you need to master a range of skills or your character will lose. If you were to ask any competent UX designer to design the perfect flow to beat a monster they would simply have a single button “Beat Monster”.That’s the paradox I was facing. Games felt like they were about sparkles and tension. Great app UX is about minimalism and simplicity. Fortunately, I found Raph Koster, the author of A Theory of Fun. Raph is known as a “Game Grammarian” and deeply deconstructs how games are made. His book is a very game-like exploration of how games work. Much like how Understanding Comics is a comic about how comics work.I met with Raph several times over 8 weeks where we took his book apart, applying it to my world of UX design. He was energetic and excited to find a parallel world that could benefit from his work. I’ve been in UX design for over 30 years and I was a bit, well, set in my ways. Raph was patient and in a good-natured way enjoyed skewering my misconceptions. This article is my journey fighting through these misconceptions.Misunderstanding 1: Just copy cool tricksThe game Horizon: Zero Dawn has an amazing heads-up-display UI. It just seemed obvious to me that there was potential for mobile apps. Raph’s point was that video games create these new UIs all the time. It would be a never ending list if I just copied what games are currently doing. His advice was to go a bit deeper and first understand how games create these types of UI.Misunderstanding 2: Games are linear and plot drivenGames have the ability to force situations, such as running into a canyon and having nowhere to go but up a ladder. Apps on the other hand, usually have the opposite, offering a broad toolkit of choices. Games, I thought, can exploit narrative to force situations which made their life easier.Raph said yes, games do use plot (they are a form of entertainment after all) but his point was that great games go so much deeper than just plot, usually working on low level mechanics first. You only have to appreciate the 12 ways to jump in Super Mario Odyssey to appreciate his point.Misunderstanding 3: Gamers practice over and over. Games have it easyRaph laughed when I said this. “You realize”, he said patiently, “that most games fail?” This made perfect sense the moment he said it. Of course, only the best games reward practice. You have to design a great game to get people to have the confidence that practicing is worthwhile.The upshot is that the Raph convinced me to forgo any quick and easy ‘cookbook of tricks’ approach to this problem and go deeper and understand better how games are built, from the bottom up. His book covers a wealth of material and I encourage you to read A Theory of Fun for more detail. But we came up with six “lenses” that cover the transition of his book to UX design.I want to be clear that video games are not exactly like applications. The goal here isn’t to copy games but instead be inspired by them. Many UX designers devour Edward Tufte’s books on information design not because they expect to visualize complex data sets themselves but more by the joyful intellectual insight that comes from understanding data patterns he describes so well. Lens 1: Story vs NarrativeThe first point Raph made felt a bit overly specific, calling out the difference between the words “Story” and “Narrative”. They felt quite similar to me but his point was that “Story” is author generated while “Narrative” is user generated. Game designers create a series of events (A-B-C) you have to pass through but what users actually do is often a surprising pattern of: staying on on step A for a very long timeskipping step B entirelycircling around step C multiple times.At first I thought he was just describing classic user testing. Of course users go through your design in ways you didn’t expect, all experienced UX designers know this. But his point was deeper: it’s not the journey, but how users recall the experience. Users will always create narrative. It’s just human nature, they will go through any experience and remember it by constructing a narrative. This is what game designers understand deeply. If your story has huge gaps and doesn’t lead the user carefully then users can’t help themselves, they will create their own narrative that fills in the gap.Most apps today will just “throw” 5 new features into an app with little connection between them. Users however, will create a narrative to help them understand what they are seeing, even if none exist. What narrative could they construct? They are likely making up stories that would frighten you. Anyone who has done user testing appreciates this. Users ‘connect the dots’ in ways that will astound, often creating mythologies that were never intended.APPSJust throw in a bunch of features into a pot.GAMESUnderstand everything is a journey. Work hard to make everything a closely connected arc of events that help the user create a narrative that matches the overall story.EXAMPLE 1:The best example of this is the original 1984 Macintosh boot sequence. Every PC of it’s day had a simple power switch which just ‘beeped’ when turned on, followed by a bunch of streaming and confusing text. The Mac booted into a startup screen showing only a pattern with a smiling Mac. Nothing more, no text, no gibberish. Eventually the desktop appeared but it was just a tiny variation, the same desktop but with an added menu bar, a disk icon and a trash can. The boot sequence started with a promise that resolved into a working desktop.Not only that, but the corners of the display were rounded which made it look more like a desktop ‘blotter’ which were popular back then.Even more, the trash can added depth to the illusion by getting ‘stuffed’ when you put something into it. The Mac took a very hardware driven concept, turning on your computer, and turned it into theater. Yes it had the boot sound, but it then showed a promise, a compromise of the final desktop and as it booted, ‘inflated’ that promise with the final working model. Why people loved the Mac is often misunderstood. I’d claim that it’s this dedication to taking people on a carefully crafted story, one which allowed users to craft a compatible narrative, that is at the heart of this devotion.EXAMPLE 2:Another more modern example is this landing page for PayPal. Notice how the page clearly invites you to choose. Are you a “Personal” user or a “Business” user? As you mouse over each section, the story unfolds, expanding your choices, offering you things you can easily understand and identify with. Each branch has a clear call to action. This is a beautiful story telling sequence that pulls you in and gets you to become an active part of the on-boarding process.Lens 2: Games are fractalRaph uses the game Frogger to explain what he means by “Games are fractal”. This single screen is composed of several levels. At first you are in a safe starter area at the bottom where you can move back and forth freely with no risk. Second is the street, where you have to dodge cars followed by a river which is a bit trickier as both the road and the ‘cars’ are moving. Finally you get into a ‘parking’ area where it’s easy to get into the first one but progressively harder as they fill up.But he literally means ‘fractal’ in that the game isn’t a simple linear “cross the road, cross, the river, and park your frog sequence. Of course, at the surface level, that’s true but his point is that it’s a nesting cascade of discoveries. To win the level you must first cross the street. To cross the street requires that you move the frog. To move the frog requires that you understand joystick timing. Each of these sub levels have their own feedback considerations:Street: the cars movementFrog: How it moves, how far it jumps each timeJoystick: Direction and speed of movement (it’s quite slow actually) Games understand that each of these levels has their own set of feedback, motivation and learning that must take place. This level of deconstruction, in a 30 year old game no less, blew my mind. Games were complex! They really paid attention to detail. There was a lot here to understand.Compare this to app designs. For very good reasons, we try very hard to fall back on guidelines. We work very hard NOT to reinvent the wheel. While this makes sense, we can’t just fall back on guidelines forever now can we? Maybe this is a bit obvious, but we’re not going to be using today’s phone UI designs 30 years from now.How do we evolve and create new UX patterns? What’s inspiring about games is that they are trying to evolve all… the… time. While clearly this would be a chaotic with commercial software, I still find