What happens when you analyze college football like the CIA?

They had a growing collection of evidence, but the difficult part was deciding what it meant. Analysts had to consider competing explanations, identify the observations that distinguished one from another, and revise their assessments as new evidence arrived. Eventually, U-2 photography provided much stronger evidence that the Soviets were installing nuclear missiles. The stakes are obviously rather different, but the analytical problem is surprisingly general. Usually there isn't one fact that gives you the answer. There are several possible futures, a huge amount of imperfect information, and a smaller number of things that actually help distinguish among them. The analyst's job is to impose structure on all of this without becoming more certain than the evidence warrants. You find versions of this problem everywhere. A foreign-policy analyst might be trying to understand whether a conflict will escalate. An investment analyst might be thinking about how geopolitics, regulation, or a new technology will affect an asset over the next decade. A government analyst might be assessing how another country will respond to a policy change. The useful question isn't simply, "What do I think will happen?" It's: What are the plausible ways this could turn out? What would have to be true for each of them? What should I be watching? And what new evidence would cause me to change my mind?

CPF starts with a strategic question and maps the range of plausible outcomes as scenarios. It decomposes the problem into the drivers and indicators that would make those scenarios more or less likely, makes assumptions explicit, and turns important uncertainties into resolvable forecasting questions. As new evidence arrives, those forecasts and the larger assessment can change with it. The idea isn't to build a crystal ball. It's to maintain a structured, explicit view of an uncertain future, and to know why your view changes when the evidence does.

We know some things with reasonable confidence, have strong opinions about others, and are almost certainly wrong about a few things we currently regard as obvious. So instead of just following the season the way I normally would, I asked Hinsley to follow Illinois the way an analyst might follow a country, company, market, or strategic issue. I started a couple months ago with the question I think most Illinois fans are ultimately trying to answer before a season: What is the ceiling this season for the University of Illinois football team? From there, I let Hinsley get to work. Its research agent began collecting information about the team: returning players, transfers, recruiting, injuries, coaching changes, position-group strengths and weaknesses, the schedule, preseason models, and so on. Its findings were that this outside view was fairly optimistic. Illinois had won 19 games over the previous two seasons, and Hinsley's research suggested a 10-win regular season and a possible College Football Playoff berth represented a plausible ceiling. But there were obvious reasons it might not happen. Illinois was replacing Luke Altmyer at quarterback, returning only one starter on the offensive line, and replacing a lot of defensive experience under a new coordinator. There was also a particularly interesting warning buried in the research: Illinois had won 13 one-score games over the previous three seasons. Maybe Bret Bielema's teams are unusually good at winning close games. Or maybe some of that was luck that wouldn't continue forever. That's exactly the kind of thing I wanted this exercise to expose. Instead of saying, "Illinois has won nine games two years in a row, so they'll probably be good again," I now had a set of assumptions hiding underneath that belief. The next step was to ask Hinsley to turn the big question into four scenarios for how the season could end, and generate initial likelihoods for each of those scenarios based on everything it knew at that point. But scenarios and even their associated probabilities by themselves aren't especially useful if you can't explain why one is becoming more likely and another less likely. So I built what we call a decomposition: essentially a map of the things that could meaningfully affect which scenario we ended up in. Mine had nine broad categories. They included the quarterback transition, offensive-line continuity and health, whether the defense could reload, performance against the best teams on the schedule, execution in toss-up games, how quickly transfers gelled, whether key players stayed healthy, special teams and possession margin, and the possibility of some unexpected roster or eligibility shock. Underneath those were much more specific things Hinsley could actually watch: Houser's completion and interception rates, sacks allowed, third-down defense, turnover margin, one-score results, injuries, and so on. This was the point where it started to feel less like having an opinion about Illinois and more like having a model of Illinois. Not a statistical model in the traditional sense, but a structured description of what would have to go right for the team to have a great season, what could prevent that from happening, and what evidence would tell me which direction we were heading. For a handful of the most important uncertainties, I went another step and turned them into forecast questions. Will Illinois allow 30 or fewer sacks this season? Will it finish with a turnover margin of at least +7? Will opponents convert fewer than 40% of their third downs? Will Katin Houser complete at least 64% of his passes while keeping his interception rate below 2.5%? Will Illinois finish in the top 12 of the final College Football Playoff rankings?