A beginning for mathematics

This essay also appears on Proofs and Prompts. Three years ago, AI systems could not reliably add two numbers. A year ago, internal models at OpenAI and DeepMind received the equivalent of a gold-medal score on the IMO. Now, these systems are autonomously resolving major open questions. It’s hard to imagine this trend continuing for another year, but I expect it will. It is clear that this will require a radical rethinking of our profession. A few weeks ago, I gave a talk titled The End of Mathematics. If you only read the title1, you might guess that this talk was about how, soon, AI will “solve” math. That’s not what it was about. The talk instead laid out a gloomy vision of the future, in which, despite the possibility of AI systems that are robustly superhuman at mathematics, the design of our institutions causes human understanding of mathematics, and possibly even mathematical progress in the abstract, to stall. I think we will avoid this future, but I also think it is plausibly the default if academic mathematics does not adapt. Despite my relative enthusiasm for the use of AI to do mathematics, I share this view with many of its detractors. Here I want to lay out, instead, a positive vision of the future of mathematics, and the human practice of mathematics. I claim we can deepen human understanding even as the production of interesting mathematics becomes less dependent on it. This essay will take as a premise that AI systems that are robustly superhuman at most or all aspects of mathematics will be here soon. But the concrete changes to our institutions I propose only require accepting the weaker premise that the production of mathematical text is becoming increasingly disconnected from mathematical understanding. What are we even trying to do here? I think it has now become clear that there is no consensus in the mathematical community as to what our goals are. Some of us want to solve problems; some of us think of mathematics as play or as poetry. For some: “Wir müssen wissen – wir werden wissen.”2 Some of us think we are penetrating the mysteries of the platonic realm. Some of us think the goal is to embody love of and understanding of mathematics,3 and to transmit that love and understanding to the next generation. My personal, if self-referential, answers are: We’re trying to produce and understand high quality mathematics. We’re trying to produce high quality mathematicians. These goals should be construed broadly. What high quality mathematics consists of has changed quite dramatically over time; we come to its definition as a community. We are not just training PhD students to do research in mathematics. A substantial part of our job, though perhaps an underemphasized one, is to educate the general public about high quality mathematics and mathematical thinking.4 Whatever our goals are, we’ve operationalized them primarily through proving theorems. Almost all papers or PhD theses have a main theorem, and ostensibly a proof of it. But it should be clear that the goal of mathematics is not to prove theorems; if it was, it would be trivial to automate. A computer or monkey could easily start at the axioms of ZFC and iteratively apply deduction rules to them, with no attention whatsoever paid to their meaning. It has had particular significance when a theorem resolves an open problem, especially one that has resisted substantial effort. Again this is easily automated; our computer or monkey can simply conjecture all mathematical propositions in alphabetical order. The general attitude of our community towards a technology that can prove theorems and solve open problems suggests that these operationalizations of our values are at best incomplete. The prospect of automating mathematics by enumerating all conjectures, and all proofs of ZFC, is probably not so disturbing to you. But let us for a moment assume the computer or monkey is very smart; perhaps it understands the results it is proving, and writes beautiful expositions thereof. Perhaps it has a good sense of what we find interesting, and is primarily focusing on those questions. Perhaps it has, in the course of enumerating theorems of ZFC, answered many of our most pressing open questions, and is asking many more fundamental open questions. Is there still a need for human mathematicians? I think so. This machine might produce answers we value, but it would not, in itself, produce human understanding of those answers. In fact I think we are at the beginning of an incredible, wonderful explosion of mathematics, and if we value human understanding, there will be more need for human mathematicians than ever before. But the profession will have to change. In the course of this change, we will have to decide what to hold on to and what to throw away. Some things I would like to preserve: learning seminars; serendipitous conversations that spark an idea; students knocking on a professor’s door to chat about math. A robust community learning exciting new mathematics. Thousands of people that, together, slowly start to resolve their confusion. I worry that much of what has been written on this topic, including some of my own past writing, focuses too much on trying to preserve the precise shape of the institutions of academic mathematics, rather than our values. How can we preserve the journal and peer review system?5 How can we protect the arXiv? How can we keep our role as gatekeepers? If you have internalized the fact that existing AI systems can produce relatively high quality results for the marginal cost of a few dollars, the idea that any semblance of the current equilibrium can survive what’s coming is absurd. As we try to find a new equilibrium, we could try to chase the edge of model capabilities. Right now AI systems arguably underperform us at theory-building, asking questions, exposition, … so we could prioritize and reward those skills. I think this is unwise: compare the speed at which the academy adapts to the speed at which model capabilities improve. We need to consider the endgame. If the models remain incapable in some domain, we can adjust later. Before I propose some relatively concrete steps we can take, let me remark on what we’re trying to protect mathematics from. There is a lot of anger at AI labs, and certain individuals at those labs. But whatever our judgment of the labs, we need a plan that does not depend on AI capabilities disappearing. The basic issue is not the labs’ behavior, ethical or not.6 It’s the technology itself. I think there is some belief that the labs will “move on” from math next year, be nationalized or broken up, or that a financial bubble will pop, somehow returning things to normal, or… But there is no way our institutions can survive unchanged when anyone with a laptop and a few hundred dollars can generate what would have been an Annals paper last year. AI does not care if you are anti-AI. Producing high-quality mathematicians The most urgent question our profession needs to answer right now is: what should our students be doing? It’s now possible to produce a PhD thesis one hasn’t even read; in terms of demonstrating understanding, mathematical text is worth the paper it is printed on.7 The value of the text no longer reliably conveys a signal about the person who produced it. In my view we should welcome interesting mathematical results regardless of provenance. But our institutions have historically relied on the same signal to indicate both mathematical progress and mathematical expertise. These now must be distinguished. I propose the following reconceptualization of the goal of a mathematics PhD: to become a world expert on some interesting, deep topic, and to be able to convey that interest and understanding to others. Part of operationalizing this might be a thesis, but the degree would be awarded primarily on the basis of a rigorous defense, in which the student explains the topic to their examiners until they are satisfied. While we might require the topic to be original, its provenance—AI or not—is irrelevant.8 How different would this look from current PhDs? I think students would still meet with an advisor, who might suggest a topic. That topic could be explored with AI assistance, or not, but the student would be responsible for understanding it; it might be much more open-ended and larger than the typical PhD is currently. The student would be trained to ask interesting questions and try to resolve them, by whatever means. To keep students on track, there might be regular meetings in which the student is asked to independently work through an unfamiliar example, apply a technique in a new case, etc. The allocative aspects of our job (hiring, graduate admissions, etc.) are in dire need of reform if we want to retain human mathematical expertise. Broadly speaking I think we should focus on rewarding skill in the parts of our jobs that cannot be automated: the internal (e.g. understanding mathematics) and social-relational parts, and operationalizations that hew as closely to those aspects of the profession as possible. For example, talks and sustained mathematical discussion now demonstrate understanding much better than papers. Once AI systems improve at exposition and “digestion,” this will be even more the case. We already interview faculty hires; we must now do the same for graduate admissions. I think we should try to foster a robust seminar culture in which speakers are expected to explain their topic to the audience’s satisfaction. Much has been written recently (by myself among others) about the fact that we are primarily interested in understanding, not merely the truth value of mathematical statements. If that is the case, let us make sure we actually understand each other.