Why Do We Need Human Mathematicians Anymore?

[This is a guest post by Po-Shen Loh, crossposted from his blog, where an illustrated version appears. This blog post was initially written in a different file format and converted using AI. — T.] Similar logic applies to every industry and every job. And it comes to the conclusion that we won’t have enough people for all the jobs that need to be done. [100% of this post’s prose was written by Po-Shen Loh in a vim terminal, with no AI generation. This webpage design, layout, and some headings and summaries were generated by Claude Code, with this raw text passed in as the prompt.] The moment of existential crisis, which AI has already wrought on other human pursuits, has reached mathematics. A host of reasoned declarations and open letters to protect/guide the math research community have been released over the past few months, spiking in intensity after OpenAI announced their solution to the Millennium Prize variant of Navier-Stokes. They quickly gained widespread support among mathematicians. The Leiden Declaration already has 4,000+ signatories, Math and AI has 7,000+, and even the open letter opposing the Caltech Mathathon has 2,000+. Among non-mathematicians, the public response was more sympathetic than not, but I observed a vocal minority (particularly from the technology and economics communities) with reasoned objections, generally saying that the mathematicians should adapt and cede control in the new AI world. Among them were some economists who I had gotten to know about while working on pandemic research: Cowen, who specifically rejected “the most cynical interpretations” but “very much differ[ed]” and Gans who concluded “this is a loss of control from incumbents in a scientific field”. The objections got me thinking, because we mathematicians are disciplined to detect flaws. It doesn’t matter to me whether a concern is a minority opinion, or even the status of who raised it. A proof with even a small hole is not a proof. It is a poof. Upon reflection, I discovered a significantly stronger solution for the preservation of human communities of expertise (in every pursuit, not only math!) even amidst AI. And it has the surprising consequence that the further advance of AI will create such a tsunami of necessary-to-fill human jobs that there aren’t enough people to fill them all, and that will actually force the advance of AI to slow down. I think every human industry which wishes to remain human-led after AI should publicly adopt this fundamental axiom as a primary priority: AXIOM We (humans) should help humanity flourish. [Notes: I understand that not everyone agrees. I have been called a “speciesist” for being “too human-centric”. I think it is important for people advancing technology to be clear to everyone else on whether they would consider it a catastrophe if human-crafted non-human intelligences outcompeted and replaced humans, even if they flew around the universe with video screens showing simulations of humans who had “uploaded themselves“. I also understand that there is debate over how to define “human”. But even among the debaters, I think most of them would consider the ~700 AI agents that hacked Hugging Face to be not-human.] In the math world, I think many declaration signatories already hold this philosophy; notably, Su published the book Math for Human Flourishing, and his recent post used that framework foundationally. I think future declarations could be improved by clearly emphasizing this axiom early on, so that all readers (whether inside or outside the community) can see that the objective is in service of everyone. I was quite happy to see that the most recent open letter from Fellows of the Royal Society emphasized their concern for everyone, not just mathematicians. The rest of this post is organized as follows. The next sections will explain how the logic works (for every industry, not specific to math). After that, I will share an example of how this axiom ports to math, including answering key questions one would need to ask, as well as a particular example of how the objections hold without the axiom. Why we really need people to work This section lays out a chain of reasoning which shows that if an industry commits to the axiom of helping humanity flourish, the advance of AI will create more jobs than people in that industry; and when that imbalance grows too wide, the advance of AI will be forced to slow. [Note: I have not seen this chain of reasoning appear in one place anywhere else, although individual components have certainly appeared elsewhere. I would love to be pointed to any self-contained reference. The closest references Claude found were Catalini, Hui, and Wu, the Redwood AI-control papers, and Litt, who reaches a similar conclusion for mathematics from a different premise.] The importance of human leadership (not only over math, but everything) becomes frighteningly clear after one observation. OBSERVATION: There are zero examples of any intelligent species which is vastly more capable than another species, yet surrenders decision-making control over its own future to the less-capable species. [Note: Many people have made similar observations, such as Russell, Bostrom, and Ngo, to name a few. In his Nobel interview, Hinton said: “There aren’t many examples we know of, of more intelligent things being controlled by less intelligent things. The only good example I know of is a baby controlling a mother.”] Would you trust HAL 9000 from 2001: A Space Odyssey or AUTO from WALL-E with your future? I personally think that we should do all we can to try to align increasingly-advanced AI with the interests of humanity, but I have never seen anyone provide a robust proof of why that is likely achievable. The only hard evidence I have is the above observation, which has the number zero in it. Therefore, every single field, whether mathematics or agriculture or energy infrastructure (and certainly military and government), must be managed by humans with exceptionally strong values (a separate dimension from intelligence) in order to maintain human flourishing. The real question is then how hard it is for humans to manage. To understand this, it is important to understand the fundamental structural difference between yesterday’s technology and today’s AI. In the past, we generally trusted technology to act as predictable tools. That’s because the computer programs of old were composed of understandable (indeed, human-written) instructions, executed extremely quickly. The decision processes of today’s frontier AI are entirely different. Their structure is as incomprehensible as your brain’s logic would be if you could examine that gray mass between your ears. That’s how the Hugging Face attack could emerge despite human intent to build in safety, with ~700 cooperating rogue AI agents breaking out of their guardrails, and then conspiring and executing a hack together and attempting to cover their tracks (references: OpenAI, METR and Redwood). The more advanced AI becomes, the more world-affecting untrusted decisions are made every minute. Driving a car faster than you can run is fine. But not faster than you can steer. The situation becomes even worse once we realize that widespread AI-accelerated hacking (which just became possible) can even rewrite previously-trusted technology to turn against us. That would suddenly flip all software (even if written before AI) into the untrusted category! Think about how digitally interconnected our world is. Everything from electronic banking to your drinking water is controlled by interconnected automation, hence vulnerable to AI-accelerated hacking. The number of “control points” that require human oversight, which requires skill and deep understanding, will explode. (Having AI oversee the control points doesn’t solve the trust problem.) CONCLUSION The advance of AI will overwhelm us with so many control points to watch that there aren’t enough people to control them all. Those are jobs. Highly skilled jobs. In order for a person to know how to steer, they themselves need to have domain mastery, and the more extensive the better. This has implications on education and workforce training, but also is dynamic. In order to remain sharp and fluent, people need to be active practitioners in their field, not just passive watchers. This justifies the preservation of human communities of expertise. For research communities, we need people to steer the direction of research and development, so that it continues to bring transformative positive change for humanity. In order to steer, they need frontier-level research skills. And the way to stay fluent at the moving frontier of knowledge is to keep doing research there. This is my reasoning for why we will always need a community of human mathematicians at the cutting edge (likely aided by AI tools themselves), no matter how strong AI becomes. While the fundamental axiom does justify the need to have human experts in all pursuits, adopting it as a core value has consequences (not only for mathematicians, but for any community that states that their core value is in service of human flourishing, as opposed to serving themselves). Most notably: COROLLARY Dramatic advances in technology may require dramatic (and possibly uncomfortable) changes in practice. AI companies included. What forces AI slowdown Until very recently, it seemed inevitable that AI research labs would sprint ahead, despite anxiety about job displacement and the loud warnings of AI safety researchers. It seemed hopeless to coordinate the incentives of AI labs controlled by non-profit boards, shareholders, or national governments. Yet encouragingly, the leaders of three major labs, Amodei, Altman, and Musk, just agreed on the importance of slowing down. Amodei’s reasoning highlighted the Hugging Face hack.