Bar Raiser Interviews, OKRs, the Spotify Model and Agile: Companies Are Full of It
Pangram verdict · v3.3
We believe that this text is a mix of AI and human-written content.
AI likelihood · overall
MixedArticle text · 1,155 words · 5 segments analyzed
On Monday I posted this on X: please i beg you all stop it with your bar raiser interview nonsense you don’t need all this to spot real talent come on That was the short version.
This is the long one. Companies are full of shit: bar raiser loops, Objectives and Key Results (OKRs), the Spotify model, sprints and standups are mostly complexity somebody copied from a bigger company without checking whether it ever worked there. In a lot of cases it didn’t and I went and checked. Then I want to show you why AI makes this urgent. And at the end, how I’d hire instead, which is how I got hired at all. What is a bar raiser interview? A bar raiser interview is a round in a hiring loop run by a trained interviewer borrowed from another team, whose job is to judge whether you’d raise the bar of the company. Amazon started it in 1999 and scores you against its 16 Leadership Principles. Amazon’s own description calls them “an objective third party during the hiring process” who “drive the pivotal decision on whether a candidate should be hired”. There are more than 10,000 of them. Revolut runs one too now: 45 to 60 minutes with a senior leader from outside the team, scored on quantified ownership. Google’s autocomplete puts “bar raiser interview” next to “revolut” and “mastercard”, which tells you how far the copy has spread. So a candidate spends an afternoon telling strangers stories in the Situation, Task, Action, Result (STAR) format about the time they showed Ownership or Bias for Action, which the strangers score against a list of 16 phrases. Then everybody copies it. Every company with a Notion page now has a “bar raiser” in the loop, a values list and a scoring rubric, as if the ritual is what made Amazon big. The best data on whether any of this works comes from a company that ran the same kind of machine at scale and then published what it found. In 2013 Laszlo Bock, who ran hiring at Google, said that when they compared tens of thousands of interview scores against how those people actually did on the job, “We found zero relationship.
It’s a complete random mess.” Brainteasers were “a complete waste of time” that “serve primarily to make the interviewer feel smart”. Of grades and test scores he said “they don’t predict anything”. And his analyst Todd Carlisle found that 4 interviews were enough to predict with 86% confidence whether Google should hire someone. Four!! Not a loop of 6 and a debrief. What does predict performance is less exciting. Paul Sackett and his colleagues re-ran the big hiring meta-analyses in 2022 without the statistical corrections that had been inflating the old numbers. The structured interview came out on top, at a correlation of 0.42 with job performance. Ask every candidate the same questions about the actual job and score the answers the same way. It takes 1 interviewer who knows the job. It doesn’t take a Bar Raiser. How well each hiring method predicts job performance, as a correlation (Sackett and others, 2022) Structured interview0.42 Job knowledge test0.40 Biodata0.38 Work sample0.33 Cognitive ability test0.31 I’ll tell you what I failed. In 2015 I was in Qatar with severe hemophilia, trying to get to any country with a healthcare system that would keep me alive, applying for engineering jobs in all of them. I got a lot of interviews and I failed all of them. Booking.com turned me down for not having enough A/B testing experience (go and look at Booking.com, the entire site is one enormous A/B test). I’d have failed a bar raiser loop too: no degree, barely any school and no STAR story about the time I showed Ownership. Then Rodrigo Paiva at McMakler, a small Berlin startup, interviewed me and hired me, with relocation. For years I told that story as “Germany was so short of engineers they’d hire anyone”. After he read my story he wrote to me to say the hire was nothing like an accident. I’d aced the technical test and he chose me because I wanted out and had shown the initiative to get there. 6 years later Spotify made that same guy a staff engineer. What is leader filtration theory? Gautam Mukunda, who taught at Harvard Business School, has a theory for what a filter like this produces. Leader filtration theory says every big organization has a process that evaluates the people who might lead it. A leader who comes up through that process is filtered: well known, thoroughly vetted and much like every other person the process would have picked. A leader who comes in through a side door is unfiltered: an outsider, a vice president who got the top job when the president died, someone nobody had the chance to screen. His claim is that filtered leaders land in the middle and unfiltered ones, in his words, “are likely to do either much better or much worse than a filtered person would”. Then he tested it on every US president. He sorted them by how much relevant experience they had before the office and how they got there, then set that against how historians rank them. 4 of the 5 highest ranked presidents were unfiltered. So were 4 of the 5 lowest ranked. Washington and Lincoln were both unfiltered and Theodore Roosevelt was a vice president who got in when McKinley was shot. Mukunda puts the result at the 99.97% significance level and the book is Indispensable: When Leaders Really Matter. How many of the 5 highest and 5 lowest ranked US presidents came in unfiltered (Mukunda, 2012) 5 highest rankedUnfiltered4Filtered1 5 lowest rankedUnfiltered4Filtered1 Now put a Bar Raiser on that.
Their whole job is to make sure nobody gets in who wasn’t vetted. By design the loop gives you the predictable middle, which Mukunda says is what you should want when things are going fine. It’s also the mechanism that keeps the Lincolns out. I’m not saying hire at random off the street. I’m saying the loop gives you a drone every time because a drone is what it’s built to give you: someone with a tidy story for every principle on the list who will follow orders in the right format. Nobody inspired got through a filter by being inspired.
Does the Spotify model work? Spotify itself never ran it. I was a staff engineer at Spotify in 2021. Squads, tribes, chapters and guilds come from a 2012 whitepaper by Henrik Kniberg and Anders Ivarsson, written when Spotify had 30 teams and about 250 people in them. Companies 10× that size copied the org chart as if it came with the music. Jeremiah Lee, a product manager at Spotify in Stockholm, wrote up in 2020 what it was like from the inside. The matrix of squads and chapters left nobody accountable.