Court To DOGE Bros: Asking ChatGPT ‘Yo, Is This DEI?’ Is Not Proper Legal Process & Also A First Amendment Violation

from the yo-chatgpt,-is-this-legal? deptBack in early 2025, DOGE bros Justin Fox and Nate Cavanaugh were handed multiple government roles with a simple mandate: go find the woke stuff and kill it. Fox and Cavanaugh had zero relevant experience for any of it — at one point Cavanaugh, a twenty-something college dropout whose main prior credential was a patent startup, was put in charge of the United States Institute for Peace. One of their assignments was the National Endowment for the Humanities, which they apparently concluded was irredeemably woke and needed to go.Their review process for millions of dollars in previously approved grants consisted almost entirely of asking ChatGPT this: “Does the following relate at all to DEI? Respond factually in less than 120 characters. Begin with ‘Yes.’ or ‘No.’ followed by a brief explanation. Do not use ‘this initiative’ or ‘this description’ in your response.” And that was it. They then relied on the results of that as a reason to cancel millions of dollars in grants that had already gone through a detailed approval process. You’ll also recall, that these two bros who probably thought themselves masters of the universe in early 2025, flipped out that the plaintiffs in the case against them, the American Council of Learned Societies, put their depositions on YouTube, where you could see them unable to define DEI, and unable to try to defend why they canceled various grants for being woke. Turns out they didn’t like facing any scrutiny themselves.Judge Colleen McMahon has now dropped a scathing 143-page ruling finding that feeding grants to ChatGPT and relying on its “why this is woke in 120 characters” output meets the “arbitrary and capricious” standard — making these cuts unlawful. Fox testified that he did not define “DEI” for ChatGPT and that he did not have the slightest idea how ChatGPT understood the term….

Nor did Fox ask ChatGPT to factor in the purpose, methodology or scholarly substance of a project – which would have required familiarity with the underlying grant materials Because the inquiry was framed only in terms of whether a project “relate[d] at all to DEI” (whatever that might mean to ChatGPT), projects whose abbreviated descriptions contained general references to “history,” “culture,” or “identity” were frequently identified by ChatGPT as relating to DEI. For example, a project to recover and analyze ancient writings attributed to Moses but excluded from the canonical Hebrew Bible and preserved in fragmentary form (e.g., the Book of Jubilees and the Testament of Moses) was classified as DEI because it claimed to “provide important insight into Jewish thought from two thousand years ago, complementary to insight from the Dead Sea Scrolls and the New Testament.”… The project description reflects a highly technical effort involving multispectral imaging, textual reconstruction, and the recovery of deteriorated source materials…. Yet, the assigned rationale consists of a single sentence linking the project to “Jewish thought,” which ChatGPT considered to be aligned with “DEI” goals. The judge highlights some other whoppers as well: nother grant supported a study of the Chinese government’s persecution of the Uyghur people. As described in the spreadsheet, the project documented surveillance practices, detention facilities, coercive assimilation policies, restrictions on language, religion, and cultural practice, and the effects of those policies on Uyghur communities in China and in the diaspora. In other words, the project concerned state policy, human-rights conditions, and the preservation of cultural and religious identity under conditions of repression by the Chinese government. Yet ChatGPT classified the project as “DEI,” Dkt. No. 248-11, at 22, apparently because it concerned the threatened erasure of a particular ethnic and religious group, albeit one located in a foreign country. Disfavoring this grant on “DEI” grounds, or any grounds, is especially difficult to square with the United States’ longstanding, bipartisan condemnation of China’s treatment of the Uyghurs, including during the first Trump Administration. Judge McMahon clearly understands that ChatGPT is a “generate approximately what you want” engine, not a tool for deeply analyzing grant applications.

The record reflects that these ChatGPT determinations were generated without any additional context beyond the cursory spreadsheet descriptions themselves. Given what courts now know about the hallucinatory propensities of ChatGPT and similar generative-AI tools, it would hardly be surprising if ChatGPT inferred, from DOGE’s repeated requests, that Fox and Cavanaugh were looking for reasons why grants could be characterized as DEI – and therefore terminable – and supplied “rationales” simply in order to satisfy the user’s perceived demand. The utter lack of reasoning behind so many of its “rationales” certainly suggests as much. This is one of the most frustrating things in lots of other stories about LLMs, in which the media and politicians and everyday users insist some sort of actual intelligence, thought, or intent behind outputs. When all any of them do is try to generate a plausible sounding response to your request. Judge McMahon understands that. And either Fox and Cavanaugh are too technically illiterate to understand that or, more likely, they didn’t give a shit. They just wanted to check boxes so they could cancel as many grants as they wanted.ChatGPT is certainly good at doing your homework for you, though.The only problem is that the law requires actual reasons and a real process for something like this — it can’t just be arbitrary. But feeding it to ChatGPT to give you the reasons why you can cancel these grants, is absolutely arbitrary. There can be no serious dispute that the review process implemented by DOGE did not conform to, or even resemble, NEH’s ordinary grant-review process, which itself was designed to comply with the mandates of its authorizing statute. The statutory process contemplates individualized evaluation by subject-matter experts, advisory review, Council involvement, and final action by the Chairperson. See supra Sections I(A), (C). And once grants had been awarded, the Act does not authorize a wholesale post-award revocation based on new ideological criteria. Rather, for awards made under § 956(c), the Act provides for post-award evaluation, financial and project reporting, and potential termination only when the Chairperson determines that a recipient has substantially failed to satisfy the purposes for which the assistance was provided or the applicable statutory criteria.

DOGE’s process, by contrast, rested on abbreviated spreadsheet descriptions and a binary ChatGPT prompt asking whether each project “relate[d] at all to DEI.” Fox then folded the ChatGPT-generated rationales into the final spreadsheets used to identify grants for termination, thereby combining DOGE’s AI-generated classifications with NEH staff recommendations. … Nothing in the record suggests that NEH had ever before employed such a process before March 2025. And nothing in the authorizing statute permitted it. There’s also the fact that DOGE has no authority to cut off these grants: DOGE had no statutory authority to terminate NEH grants. And on the undisputed evidence, DOGE – not the NEH Chairperson or anyone else at NEH – effectuated the terminations at issue here. This is not a case involving a routine disagreement over how to interpret a statute. Nonstatutory ultra vires claims are reserved for extraordinary cases in which “the agency action go[es] beyond mere legal or factual error and amount[s] to a clear departure by the [agency] from its statutory mandate.” … They require a showing that the agency “plainly acts in excess of its delegated powers and contrary to a specific prohibition in the statute that is clear and mandatory.”…. This case presents something more basic. It is not that DOGE misconstrued a statutory provision conferring authority on it; it is that Congress conferred no authority on DOGE at all with respect to the awarding, continuation, or termination of NEH grants. The Supreme Court has made it clear that Congress — not the executive branch — has the power here. (Amusingly, the GOP loves to point this out, but only when Democrats are in the White House.) The Executive Orders and DOGE-directed termination process did not carry out the NEH grantmaking scheme Congress prescribed by law in a duly enacted statute. They displaced it. Congress vested funding authority in the NEH Chairperson. DOGE substituted a presidential policy judgment for a statute duly enacted by Congress.

That is not faithful execution of law Judge McMahon even points out that the Trump administration’s posturing regarding DOGE in other cases is part of what sinks it in this case: As the Solicitor General represented to the Supreme Court, “USDS [DOGE] is a presidential advisory body within the Executive Office of the President” and “USDS has no organic statute and is not created by Congress.” Appl. to Stay Pending Certiorari or Mandamus and Request for Immediate Administrative Stay at 5, 21, In re U.S. DOGE Serv., No. 24A1122 (emphasis added). If DOGE “has no organic statute and is not created by Congress,” then it necessarily lacks any statutory delegation of authority to terminate NEH grants. And it certainly cannot exercise such authority in derogation of the very purposes that Congress wrote into the agency’s authorizing statute. The court goes on to explain that an executive order simply cannot overrule Congress, but more to the point, Trump’s executive order setting up DOGE in the first place itself admits that it doesn’t have the powers it sought to exercise in cancelling these grants. … the Executive Orders, read on their own terms, confirm the absence of any operative grant-termination authority in DOGE. Executive Order 14158, Establishing and Implementing the President’s “Department of Government Efficiency,” first creates the principal DOGE entity by renaming the United States Digital Service as the “United States DOGE Service (USDS)” and providing that it “shall be established in the Executive Office of the President.” Exec. Order No. 14158 § 3(a), 90 Fed. Reg. at 8441. It then separately creates, “within USDS,” a “temporary organization known as ‘the U.S. DOGE Service Temporary Organization’” and states that this temporary organization is created “in accordance with section 3161 of title 5, United States Code.” Id., § 3(b), 90 Fed. Reg. at 8441. But the only reference to a statute in the Order – to 5 U.S.C. § 3161 – attaches to the temporary organization, not to the principal DOGE entity.