Financial institutions, like many businesses today, are intrigued by the possibilities of generative AI.

However, like many other organizations, banks are also struggling with outdated tech stacks that can hamper their ability to run this emerging technology in a way that is effective, according to Malcolm DeMayo, global VP, financial services industry for chipmaker NVIDIA.

“The conversation right now is really around helping financial firms to get prepared,” DeMayo said in an interview.

Paving the way for generative AI

NVIDIA has been investing heavily in the AI space for over a decade at this point, and financial firms are also “not new” to adopting innovative technologies, DeMayo said.

However, what makes generative AI a uniquely transformative technology when compared to other, earlier versions of AI is it is “standing on the shoulders of giants, and standing on the shoulders of the PC wave, the internet wave, the mobile wave, the cloud wave,” DeMayo said. “And it's now available to 7 billion people out of 8 billion people on the planet, completely democratized.”

NVIDIA works with financial institutions like Capital One to “accelerate the machine learning end to end pipeline,” DeMayo said.

The chipmaker, which swiftly emerged as a key player in the fast-growing generative AI space, has made headlines as investment in the new technology grows. Generative AI is helping to drive “exponential growth” in the computing space, NVIDIA CFO Colette Kress said in May during the company’s most recent earnings call. For its fiscal 2024 first quarter, the Santa Clara, California-based company reported total revenue of $7.19 billion, CFO Dive previously reported.

Since the recent dawn of OpenAI’s ChatGPT last November — which was trained on NVIDIA’s tech, DeMayo said — NVIDIA has reached a market capitalization of more than $1 trillion this year, after seeing their stock price more than triple, according to a Tuesday report by the Financial Times.

However, the world is still in the “early innings” of generative AI, he said — and banks especially face challenges to implementing the technology with their present infrastructure, he said.  

“This new generation of AI doesn't run well, at all, on the current computer infrastructure that banks have,” DeMayo said. “CPU servers just don't have enough gas.”

DeMayo described a recent conversation with a banking professional who described their infrastructure as, “they have all the geological layers of time,” he said. “So it's like, you can look at the layers and you can kind of see, okay, ‘this was this decade, this was that decade.’”

Before implementing generative AI, banks and their leadership team need to be able to innovate their outdated tech stacks and servers, which requires making the jump to accelerated computing, DeMayo advised.

Accelerated computing uses specialized hardware that helps to speed up various tasks through parallel processing, which bundles frequent tasks together, a 2021 blog post by NVIDIA explained. Financial firms must either move to the cloud where they can access this as a service — and pay as they go — or build such a base on premise, which can allow them to take advantage of the dawning generative AI age.

Boosting productivity with generative AI

Despite the increased attention — and investments — being paid to generative AI, many companies and banks have yet to budget for the necessary steps required to implement it, DeMayo said.

“If you’re a CFO, you're licking your chops, and you're thinking, ‘what do I have to do to take advantage of this?’…it comes back to making sure you're recruiting data scientists, making sure that you're building a high quality domain specific data set, and that you have access to accelerated computing, either on premise, in cloud or both,” DeMayo said.  

CFOs, who play a critical role when it comes to adopting new technologies, should look at generative AI as a way to both reduce costs and as a “way to improve productivity,” Demayo said. Many are looking for ways to improve efficiency at their organizations as economic headwinds continue to affect their businesses’ bottom line.  

For example, companies can tap generative AI to complete tasks much more quickly — both reducing one’s people cost as well as freeing up key employees to turn their attention to other, more noteworthy tasks. CFOs at banks, for example, could take advantage of generative AI models trained to act like an assistant, a junior wealth manager or a junior call center agent, lessening the number of time-consuming manual tasks that employees need to complete while also cutting down on costs.  

“The opportunity to make your people more productive is enormous,” DeMayo said. “That's how generative AI is different.”