Navigating the rising costs of AI inferencing

In 2025, the worldwide expenditure on infrastructure as a service and platform as a service (IaaS and PaaS) reached $90.9 billion, a 21% rise from the previous year, according to Canalys. From I’m seeing, this surge is primarily driven by companies migrating their workloads to the cloud and adopting AI, which relies heavily on compute resources. Yet as businesses eagerly embrace these technologies, they are also encountering obstacles that could hinder their strategic use of AI.

Transitioning AI from research to large-scale deployment poses a challenge in distinguishing between the costs associated with training models and those linked to inferring them. Rachel Brindley, senior director at Canalys, notes that, although training usually involves a one-time investment, inferencing comes with expenses that may vary considerably over time. Enterprises are increasingly concerned about the cost-effectiveness of inference services as their AI projects move towards implementation. It is crucial to pay attention to this, as costs can quickly add up and create pressure for companies.

Today’s pricing plans for inferencing services are based on usage metrics, such as tokens or API calls. As a result, companies may find it difficult to predict their costs. This unpredictability could lead businesses to scale back the sophistication of their AI models, restrict deployment to critical situations, or even opt out of inferencing services altogether. Such cautious strategies might hinder the overall advancement of AI by constraining organizations to less cutting-edge approaches.

The effects of busted budgets

The concern about inference-related expenses is justified, as several businesses have experienced the consequences of overestimating their requirements and incurring high bills as a result. A notable example is 37signals, which operates the Basecamp project management tool and faced a cloud bill surpassing $3 million. This unexpected discovery led the company to transition its IT infrastructure management from the cloud to on premises.

Organizations are increasingly aware of the risks associated with utilizing cloud services like never before. Gartner has cautioned that companies venturing into AI adoption could encounter cost estimation discrepancies ranging from 500% to 1,000%. These may stem from hikes in vendor prices, overlooked expenses, and mismanagement of AI resources. As businesses explore the potential of AI technologies, miscalculating their budgets is a significant risk to innovation and progress.

Exploring different hosting options

Many organizations are now reassessing their approaches to the cloud. As companies increasingly depend on public cloud services, such as Amazon Web Services (AWS), Microsoft Azure, and Google Cloud, some are turning towards specialized hosting providers or colocation services. These options offer a pricing structure that could improve resource handling, enabling businesses to enhance their AI applications without worrying about unexpected costs.

Currently, the top cloud services companies hold a significant market position, with more than 65% of customer expenditure under their control. However, AWS has experienced a decrease in growth, from 17% in the most recent quarter compared to 19% in the preceding one. On the other hand, Microsoft and Google have maintained growth rates surpassing 30%. Businesses are looking for cost-effectiveness and tailored solutions offered by specialized providers.

Cloud service providers are aware of the challenges associated with inference expenses and are actively investigating methods to enhance effectiveness and lower service charges. According to Canalys, specialized expertise in AI tasks might be crucial in alleviating the burden of inferencing expenses by integrating tailored hardware accelerators alongside GPUs, optimizing efficiency and reducing costs.

Despite these efforts to implement AI on a large scale in public cloud environments, there are still doubts about its long-term sustainability. Alastair Edwards, chief analyst at Canalys, has highlighted the risks of using AI in the cloud, noting that costs could become unmanageable as organizations expand their AI projects. This poses a challenge for businesses seeking to ensure the long-term success of their AI initiatives.

Practical steps to controlling inferencing costs

In addressing the challenges faced by businesses today, it’s essential to take a proactive stance towards controlling inferencing expenses. I recommend considering the following  approaches:

Use tools that offer real-time insights into how resources are being used and how money is being spent. By monitoring cloud usage patterns, organizations can make informed choices on where to expand and where to save.

Perform cost estimations to predict costs depending on different usage trends, aiding in forecasting expenses and preventing budget overages.

Select the pricing model wisely by comparing options offered by cloud providers. Usage-based pricing may not be the best choice in all cases; fixed pricing could be more suitable for specific organizational requirements.

Consider a combination of public and private cloud resources. A hybrid cloud can enhance flexibility and optimize costs effectively.

Working together with cloud service providers can help you uncover ways to manage costs effectively and efficiently. Providers often offer customized solutions designed to address industry-specific challenges.

The path to successfully integrating AI into the business world is full of obstacles, especially when it comes to controlling the expenses of making inferences in cloud setups. As businesses increasingly incorporate AI solutions into their operations, it becomes crucial to prioritize cost-effectiveness and practices. Being aware and taking steps to address these hurdles are important for companies to fully leverage the power of AI and spur creativity in their industries. Don’t wait for an unexpectedly high bill before you decide to act.