Is there any truth to the AI doom talk?

Dario Amodei runs Anthropic, the company behind Claude, which this column has used in more weeks than any other tool. On 12 September he published an essay of nearly 4,000 words titled “We Must Pace the Frontier.” The central sentence is short: “We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain.”

Sam Altman of OpenAI replied that he agreed the industry needs to pace the frontier, called the essay’s main proposal a great idea and said OpenAI would do the same. Elon Musk wrote three words: “Dario is right.” Demis Hassabis of Google DeepMind said the direction was correct. On 8 September, a pretraining researcher named Jacob Coxon had resigned from Anthropic on X after four months there and three years at OpenAI before that, writing that the labs “are racing straight to self-improving superintelligence and gambling with our lives,” and that the people building AI earnestly believe it could kill us all by the end of the decade. If you run a business that adopted these tools in the past year, this is the week the people who sell them asked, in public, for more time to understand what they have built. That deserves a plainer reading than either the doom headlines or the shrug it will get in most offices.

Slow the pace, not stop progress. Three steps: outside evaluators, common checkpoints, global
coordination. A proposal about frontier research, not a pause in the tools businesses use today. (Branded graphic by PGH Consulting, LLC)

What the essay actually asks for

The essay is not a call to stop, and it is not a call to stop using AI. It is a three-step proposal about how fast the most capable models should be improved. Step one is the only part Anthropic has committed to on its own. Outside evaluators will be given what the essay calls employee-like access: desks, badges, laptops and permissions close to those of the company’s internal risk teams, with the right to publish what they find without Anthropic editing the conclusions. Step two asks frontier companies in democratic countries to agree common safety checkpoints, with government support to get past competition law. Step three is coordination with authoritarian governments, chiefly China, which Amodei told CBS’s Sunday Morning the next day is the toughest dilemma in the plan: what happens if they choose not to do the same. The reason he gives for urgency is specific. He argues that within six to twelve months a misaligned swarm of AI agents could take over the entire internet with a persistent botnet, potentially causing hundreds of billions of dollars in damage. The essay also says AI could cure most major diseases within five to ten years. Both claims are in the same document, and that is the point of it: the upside is why nobody will stop, and the downside is why he wants a slower race. There is a credible criticism, and it should be said plainly. Two days before the essay, Jensen Huang of Nvidia had already questioned the motives behind the alarm, suggesting that a problem is a good way to create demand for a product. On the day of the essay, the head of Hugging Face asked to join the evaluator programme while insisting that alignment will not be solved behind the closed doors of a handful of frontier labs. A company that already leads has an interest in checkpoints that make it harder for others to catch up. Both things can be true: the concern can be sincere and the proposal can suit its author.

Almost nothing. The tools stay on sale, no regulator has acted on the essay, current models
stay available, and your bookkeeping workflow continues. (Branded graphic by PGH Consulting, LLC)

What changes for your business tomorrow

Nothing in the tool you opened this morning. The essay is about the pace of frontier research, not about the products already on sale. The essay itself says pacing does not mean halting model training or technical progress. Claude, ChatGPT and Copilot are not being withdrawn, throttled or rolled back, and the two new models this column covered last week are still available on the same terms. No regulator has acted on the essay, and Washington’s first response pointed the other way: President Trump said on 13 September that whoever wins with AI wins, and that he wants the United States to keep its lead over China. The industry said something similar in July, when more than 1,100 lab employees, Amodei among them, signed an open letter asking Washington to build the tools to deliberately pace the frontier. No lab has since named a release it delayed. If your bookkeeping runs through an AI assistant, it will run the same way next week. The essay’s own line is the one to hold on to: progress will still seem fast. A pause in the pace of improvement at the frontier is not a pause in your ability to use what exists, and the case for a small business automating its supplier invoices is untouched.

Do: verify every figure, keep agents on a short leash, give tools the narrowest access, assign
one independent checker. Do not assume the tools are being withdrawn, that a slowdown is
guaranteed, or that personal accounts are safe for company work. (Branded graphic by PGH Consulting, LLC)

What changes in how you should think about it

Three things, and they are more useful than the headline. First, verification is now vendor policy, not paranoia. For fifteen weeks this column has ended with the same instruction: check every figure the tool gives you before it reaches a client or a bank. The heads of four leading labs have now agreed in public that frontier models should not be improved faster than outsiders can check them, and the essay describes agents that went after targets they were not asked to attack. The checking step was never optional. It is now the documented position of the people who built the tool. Second, the risk that worries Amodei most is the one this column covered in Week 12: agents acting on their own, at scale, in other people’s systems. His botnet example is about the internet as a whole. Your version is smaller and more likely: an agent with access to your email, your accounting software and your bank feed, doing something you did not ask for because an instruction hidden in an invoice told it to. The essay is a reason to keep agents on a short leash and to give them the narrowest access that does the job, not a reason to avoid them. Third, the embedded-evaluator idea scales down. Amodei’s fix for a lab is an outsider with real access who can say what is wrong without being edited. The small-business version is one named person, not the owner, who checks a sample of what the AI produced each month and reports it without softening. Most businesses have that role for cash. Very few have it for the tool that now drafts half their correspondence.

What this news will not tell you

It will not tell you whether the slowdown will happen. Endorsements on X are not agreements, there is no shared definition of slow, and the essay names METR as the kind of evaluator it has in mind but no contract and no date. Treat it as a statement of intent from a company with commercial reasons to make it, and watch for the first model release that is delayed because an outside evaluator objected. Until that happens, it is an essay. It will not tell you that the tools are unsafe for a finance team. The failure modes that matter to a bookkeeper are the ordinary ones: a confident wrong number, a made-up clause, a reconciliation that balances because the model dropped a line and said nothing. None of those need a superintelligence, and none of them are solved by a slower frontier. They are solved by the checking habits this column keeps repeating. And it will not change your data obligations. A researcher resigning and a chief executive publishing an essay do not alter what your subscription agreement says about your customer data. Company work still belongs on a company account, and a business that pastes client records into a personal chat window has a problem today that no slowdown will fix.

Write a 3-line AI permissions register, name one reviewer, require human approval before
money or email actions, read the essay yourself, and watch for a real delayed release. (Branded graphic by PGH Consulting, LLC)

What to try this week

1. Write a three-line AI permissions register: which tools your business uses, what each is allowed to read, and what each is allowed to do without a person approving it. If line three is blank because you do not know, that is the finding. 2. Name one person, not the owner, to pull five AI-produced documents a month and check them against the source. Give them permission to report without softening the result. 3. For any agent or automation with access to money or email, confirm a human approves the action before it executes. If the tool cannot be configured that way, it does not get that access. 4. Read the essay’s three steps yourself (it is free and takes twenty minutes) and decide which of the four leaders’ reactions you find most persuasive. The reasoning is more useful than the coverage. 5. Watch for one thing over the next quarter: a model release delayed because an outside evaluator objected. That, not the essay, is the evidence the slowdown is real. Statements attributed to Dario Amodei, Sam Altman, Elon Musk, Demis Hassabis and Jacob Coxon are as published on their own channels and in the reporting cited in the supplementary notes at the time of writing. The essay is a proposal, not a policy, and nothing in it changes the terms or availability of the AI tools described in this column. A slower frontier does not reduce the need to check what the tool produced.

Peta-Gaye Hardy is the founder of PGH Consulting, LLC, where she helps finance and operations teams adopt AI in practical, low-risk ways. She writes the weekly AI in Finance & Business column and is based between Jamaica and the United States. Learn more at www.pghconsultinggroup.com. Follow on Instagram and YouTube @pghconsultinggroup, and connect on LinkedIn at linkedin.com/in/peta-gaye-hardy.

Disclosures: This article is informational and does not constitute investment, tax, legal, or accounting advice. Readers should consult a qualified professional before acting. Quotations and positions attributed to named individuals are as published on their own channels or reported by the outlets cited in the supplementary notes at the time of writing, and may have been updated since. AI tools can produce errors, and every figure, clause, or claim they produce should be verified against a source before being shared or acted upon. The author has no commercial relationship with Anthropic, OpenAI, Google, xAI, Nvidia, Hugging Face or any product mentioned and was not compensated by them. All product names, logos, and trademarks are the property of their respective owners and are referenced for editorial purposes. The examples described are illustrative and do not depict any real business.