Cloud & GenAI are causing ‘bill shock’ & project failures: GPUs to the rescue

As organizations put their hands together with cloud computing and Gen AI, data analytics and powerful business insights are within reach for large enterprise organizations. However, that very combination has become responsible for producing massive and ever-increasing volumes of data, causing IT costs to exponentially increase at an unsustainable and unprecedented rate.

Organizations continue to pour money into AI for analytics, a process that’s proven to bring down costs and save time, but the ground reality is something else. According to the 2024 State of Big Data Analytics Report by data analytics company SQream, there is a growing disconnect for enterprises between the cost of analytic projects and the operational value being realized. This highlights the pressing need to change the way companies handle huge volumes of data to reduce ‘bill shock’ and avoid the risk of project failures.

Read more: How Gen AI impacts IT: “The happiness of engineers is affected in very different ways by Gen AI, depending on their seniority”

SQream surveyed 300 senior data management professionals from US companies with at least US$5 million annual spend on cloud and infrastructure. Despite the already substantial budgets at play, 98% of these companies still experienced ML project failures in 2023.

Adding more compute power has in the past been the go-to way to yield better AI results. However, SQream’s survey highlights that doing so indefinitely is an untenable approach for modern, data-driven enterprises, with the complexity of queries and volume of projects being compromised due to skyrocketing bills and IT costs.

Deborah Leff, Chief Revenue Officer of SQream spoke to The Tech Panda about how in the light of these survey results, leaders are starting to recognize the transformative power of GPU acceleration.

Machine learning is constant learning. It’s data constantly coming in and retraining those models. And I think as data gets bigger, complexity grows, because analytics begets analytics. And so companies are getting more and more sophisticated in what they can do with analytics

Deborah Leff, Chief Revenue Officer of SQream

“The immense value of an order-of-magnitude performance leap is simply too valuable to be ignored in the race to become AI-driven,” she says.

A look at the survey statistics tells a compelling story of how things are going wrong for enterprises. For example, although billing cycles vary from company to company, when asked how often they experience bill shock, 71% of respondents (more than 2 out of 3 companies) reported they are surprised by the high costs of their cloud analytics bill fairly frequently, with 5% experiencing bill shock monthly, 25% bimonthly and 41% quarterly.

“That’s out of control. Though it doesn’t mean it threatens the organization, it does mean that the organization is continuously blindsided by overspend that they’re not anticipating spending,” says Leff.

As with data analytics, the cost-performance of ML projects is key to successful business predictions. However, given that in ML the more experimentation a company conducts, the better the final result, it is no surprise that 41% of companies consider the high costs involved in ML experimentation to be the primary challenge associated with ML and data analytics today.

“Machine learning is constant learning. It’s data constantly coming in and retraining those models. And I think as data gets bigger, complexity grows, because analytics begets analytics. And so companies are getting more and more sophisticated in what they can do with analytics,” she states.

98% of companies experienced ML project failures in 2023, the top contributing factor being insufficient budget (29%). Other top contributing factors were poor data preparation (19%) and poor data cleansing (19%).

Why is this happening?

“Part of the reason why these cloud bills are growing is because the CPU has a limited number of compute cores internally. And so you have to spin out all of these extra resources to churn through large data sets or complex queries,” Leff explains.

A CPU is the brains of the computer, it’s what we run data processing on. But Leff points out that the ecosystem here is 40 years old.

“It was never designed for what we can now do,” she says.

The solution, she says, lies in going ahead with the right combination of technology. Recently, there has been a huge spike in GPUs, which in comparison to CPUs, can have tens of thousands of cores. This means they can do an awful lot of tasks all at the same time and have a massive acceleration effect.

It’s no wonder that 3 in 4 executives are looking to add more GPUs in 2024. 75% of those surveyed said that adding GPU instances to their analytics stack will have the most impact on their data analytics and AI/ML goals in 2024.

But close to half of the respondents admitted they compromise on the complexity of queries in an effort to manage and control analytics costs, especially in relation to cloud resources and compute loads. 92% of companies are actively working to “rightsize” cloud spend on analytics.

SQream creates a novel way of leveraging GPU and CPU resources together, so that the things that only a CPU can do gets done with CPU resources. But the things that can be accelerated by leveraging GPU for resources, it leverages those and so on.

“You can leverage much more of the GPU and you can take something that took four hours and you complete it in two minutes,” she says.

Read more: AI makes customer service cooler but still needs the human touch

“To get ahead in the competitive future of AI, enterprises need to ensure that more big data projects reach the finish line. Constant compromising, including on the size of data sets and complexity of queries, is a huge risk factor that corporate leaders need to address in order to effectively deliver on strategic goals,” Matan Libis, VP Product at SQream, said in a press release.

In the somewhat distant future, when unimaginable amounts of data face enterprises, quantum computing will step in. But till then, GPUs are saving the day.