A New York federal ruling has turned DOGE's ChatGPT grant purge into a warning for every founder selling AI into government. Efficiency is not a legal defense when the workflow cannot survive basic scrutiny.
The federal government just got a clear reminder that asking ChatGPT to help decide who keeps public money does not make the decision less accountable. U.S. District Judge Colleen McMahon ruled in the Southern District of New York that the Department of Government Efficiency acted unlawfully and unconstitutionally when it helped terminate more than $100 million in National Endowment for the Humanities grants tied to what officials considered DEI.
The ruling matters beyond the universities, museums, libraries, writers and research groups that lost funding. It is one of the clearest courtroom examples yet of what happens when an AI workflow is dropped into a public decision without a defensible policy, a reliable record, or people who can explain what the system actually did. For civic-tech founders and AI compliance startups, this is no longer a theoretical risk. It is now litigation history.
As The Washington Post reported, McMahon found that DOGE did not have authority to cancel congressionally approved NEH grants and that the selection process violated the First Amendment and the equal protection component of the Fifth Amendment. The plaintiffs included the American Council of Learned Societies, the American Historical Association, the Modern Language Association and the Authors Guild, whose members challenged the April 2025 cancellation of more than 1,400 grants.
The facts that made the ruling so damaging were not buried in a complex machine-learning system. DOGE staffers Justin Fox and Nate Cavanaugh relied on ChatGPT to screen grant descriptions for links to diversity, equity and inclusion. The prompt asked whether a grant related at all to DEI, required an answer in less than 120 characters, and told the system to begin with a yes or no followed by a short explanation.
That might look efficient in a spreadsheet. In court, it looked reckless. The judge focused on the absence of a stable definition of DEI, the use of protected characteristics as detection signals, and the lack of meaningful human review before grants were marked for termination. Terms connected to race, sexuality, national origin, religion and Indigenous communities became flags in a process that was supposed to evaluate public funding under law, not keyword suspicion.
The result was predictable. Grants involving Holocaust education, civil rights, Indigenous culture, HIV in prisons, collections management after natural disasters, preservation training and even an HVAC project were swept into a termination process that officials struggled to justify later. Discovery showed that DOGE's review covered 1,163 grant proposals, flagged 1,057, and kept only 42. That is not careful triage. It is mass classification dressed up as administrative judgment.
McMahon also rejected the idea that the government could distance itself from the chatbot's output. If public officials use an AI tool as part of an official decision, the decision remains theirs. That point should land heavily in Washington's AI market, where many vendors still pitch automation as a way to reduce friction without fully confronting the legal duties that come with public power.
Why Founders Should Pay Attention
For startups selling into agencies, the ruling changes the conversation. It is no longer enough to promise faster reviews, lower costs, or fewer hours spent by staff. Any model-assisted decision that affects grants, benefits, permits, enforcement, hiring, immigration, education, health care, or contracting needs a record that explains what data was used, what policy standard applied, who reviewed the output, and how an affected party can challenge the result.
That creates risk for thin AI wrappers built around general-purpose models. A chatbot prompt, a spreadsheet and a few human approvals may be good enough for an internal brainstorm. They are not good enough when federal funding is being cut. Public agencies need systems that preserve inputs and outputs, document decision rules, flag uncertainty, support appeal rights and make it possible for supervisors, inspectors general and courts to reconstruct what happened.
There is also an opportunity here. Compliance infrastructure for government AI is likely to become more valuable, not less. Startups that can help agencies create audit trails, bias testing, policy mappings, procurement records and human-in-the-loop review may find a stronger market after this ruling. The pitch has to change, though. The strongest product is not the one that claims to replace public officials. It is the one that helps them prove they followed the law.
The decision may not become a broad ban on algorithmic grant review. Courts tend to rule on the facts in front of them, and this record was unusually stark. But it could become a practical precedent for how plaintiffs challenge AI-assisted government action: demand the prompts, identify the classification logic, test the statutory authority, and ask whether the agency treated a model's answer as a substitute for reasoned decision-making.
The government is likely to appeal, and McMahon's order does not require immediate payment of every dollar. It does require the government to rescind the termination letters, which puts the NEH grants back onto a legal footing and forces officials to confront the process they used. For founders, the lesson is already available. Selling AI to government is not just a software sale. It is a promise that the system can operate inside constitutional, statutory and procedural boundaries. The next wave of public-sector AI will be judged less by how fast it moves and more by whether anyone can defend what it did.
Also read: SafePal faces a trust test after a reported customer data leak • A smaller Z-Image text encoder lowers the cost of image AI • The world is trying to log off U.S. tech and founders need to take that seriously
