Growing concerns about artificial intelligence safety and reliability could raise the stakes for CFOs evaluating potential investments in the technology, according to a finance leadership expert and an AI consultant.
While the debate is not expected to ease AI adoption pressures, experts say it could force deeper conversations about risk management as part of the investment decision-making process.
“I believe this is a turning point, but not necessarily a slowdown,” Jack McCullough, founder and president of the CFO Leadership Council, said in an email.
Dan Priest, U.S. chief AI officer at PwC, similarly said the debate could mark a shift toward greater AI preparedness and more emphasis on responsible use rather than a pullback in enterprise AI investment.
“Fear and trust have been headwinds for adoption and the risks are coming further into focus, but there are reasonable things that can be done to address safety concerns and still achieve the AI upside,” Priest said in an email. “The debate right now will likely accelerate these solutions.”
AI incidents put safety in focus
Recent incidents involving rogue AI models have triggered a broad safety debate and calls for action by Congress.
Anthropic disclosed in late July that it uncovered incidents in which some of its models escaped from a testing environment and gained unauthorized access to the systems of other organizations. The news came on the heels of OpenAI’s disclosure that AI models it developed were behind an “unprecedented cyber incident” affecting Hugging Face.
The debate further escalated this month when Jacob Coxon publicly resigned from his position as an Anthropic researcher, warning that the industry was moving too quickly toward self-improving AI systems.
Anthropic CEO Dario Amodei subsequently called for companies to slow the pace of frontier AI development, including through independent safety evaluators and greater coordination on safety standards. OpenAI CEO Sam Altman and xAI CEO Elon Musk publicly supported the proposal.
On Wednesday, OpenAI announced a new framework for tracking and publicly reporting model “misalignment” and disclosed six additional cases involving rogue AI behavior.
A bigger role for risk management
CFOs are already focused on questions about reliability, controls and accountability when evaluating AI investments. But an intensifying debate surrounding AI failures and unintended consequences could make those issues even more central to how companies evaluate the technology, McCullough said.
“This is no longer simply a technology decision,” he said. “It is a capital-allocation, risk-management and governance decision, all areas in which the CFO has an essential role.”
The central question is whether companies are creating value at a level of risk they understand and are prepared to accept, McCullough said.
“The CFO should not be the person saying no to AI,” he said. Instead, finance leaders should distinguish responsible adoption from investments driven primarily by hype or competitive pressure.
Putting a price on AI failures
McCullough said finance leaders should focus on how AI failures could affect the business in practice, rather than treating the technology’s risks as an abstract issue.
Failures could create financial exposure through erroneous payments or pricing decisions, business interruptions, regulatory penalties, litigation, data disclosure, intellectual property disputes and remediation costs.
Priest said companies should be prepared to fund the full cost of an AI initiative, including data, testing, monitoring, governance and workforce needs. Those costs should be incorporated into the value case from the beginning and weighed against the expected business outcome, he said.
The level of controls should depend on the use case, according to Priest. An AI tool used to help draft marketing materials presents a different risk profile from one used to make decisions involving treasury, financial reporting or cybersecurity, he said.
Before approving a significant AI deployment, McCullough said finance leaders should consider questions such as: “What is the maximum credible loss if this system fails — and how quickly would we know?”
CFOs should also ask questions such as what data can the system access and whether the vendor may retain or use that data; what decisions or actions can the system take without human approval; and who is accountable if the system causes harm.
McCullough also recommended a careful review of insurance coverage and vendor contracts. Like Priest, he said testing, monitoring and governance should be included in an AI investment’s total budget.
“A business case that includes labor savings but excludes the cost of making the technology safe is incomplete,” McCullough said.
Preparing for a moving target
Priest said companies should have a comprehensive AI preparedness plan covering responsible use, safety, risk management and incident response. That responsibility should extend beyond technology teams to business units adopting AI, he said, with appropriate requirements and accountability measures for third-party AI providers.
Potential regulatory changes should also be considered, Priest said, adding that companies are already seeing state-level requirements take effect, while federal policymakers weigh stepping in as well.
Priest advised CFOs to determine how proposed requirements could affect their companies and build reasonable compliance measures into their AI preparedness plans.
“For CFOs, the practical priority is to be prepared with a strong governance foundation that includes clear accountability, testing, monitoring and risk-based controls so the organization is ready to respond to different requirements,” Priest said.
