Conquering the costs and complexity of cloud, Kubernetes, and AI

Platform engineering teams are at the forefront of enterprise innovation, leading initiatives in cloud computing, Kubernetes, and AI to drive efficiency for developers and data scientists. However, these teams face mounting challenges in managing costs and complexity across their expanding technological landscape. According to industry research conducted by my company, Rafay Systems, 93% of teams face hurdles in Kubernetes management, with cost visibility and complex cloud infrastructure cited as top challenges for organizations.

While IT leaders clearly see the value in platform teams—nine in 10 organizations have a defined platform engineering team—there’s a clear disconnect between recognizing their importance and enabling their success. This gap signals major stumbling blocks ahead that risk derailing platform team initiatives if not addressed early and strategically. For example, platform teams find themselves burdened by constant manual monitoring, limited visibility into expenses, and a lack of standardization across environments. These challenges are only amplified by the introduction of new and complex AI projects. There’s a pressing need for solutions that balance innovation with cost control so that platform teams can optimize resources efficiently without stunting modernization.

The problem with platform team tools

Let’s zoom out a bit. The root cause of platform teams’ struggles with Kubernetes cost visibility and control often traces back to their reliance on tools that are fundamentally misaligned with modern infrastructure requirements.

Legacy cost monitoring tools often fall short due to a multitude of reasons:

They lack the granular visibility needed for cost allocation across complex containerized environments.

They weren’t designed for today’s multi-team, multi-cloud architectures, creating blind spots in resource tracking.

Their limited visibility often results in budget overruns and inefficient resource allocation.

They provide inadequate cost forecasting and budgeting.

Our research shows that almost a third of organizations underestimate their total cost of ownership for Kubernetes, and that a lack of proper visibility into costs is a major hurdle for organizations. Nearly half (44%) of organizations reported that “providing cost visibility” is a key organizational focus for addressing Kubernetes challenges in the next year. And while standardization is essential for effective cost management and successful overall operational efficiency, close to 40% of organizations report challenges in establishing and maintaining enterprise-wide standardization—a foundational element for both cost control and operational efficiency.

Platform teams that manually juggle cost monitoring across cloud, Kubernetes, and AI initiatives find themselves stretched thin and trapped in a tactical loop of managing complex multi-cluster Kubernetes environments. This prevents them from driving strategic initiatives that could actually transform their organizations’ capabilities.

These challenges reflect the overall complexity of modern cloud, Kubernetes, and AI environments. While platform teams are chartered with providing infrastructure and tools necessary to empower efficient development, many resort to short-term patchwork solutions without a cohesive strategy. This creates a cascade of unintended consequences: slowed adoption, reduced productivity, and complicated AI integration efforts.

The AI complexity multiplier

The integration of AI and generative AI workloads adds another layer of complexity to an already challenging landscape, as managing computational costs and the resources it takes to train models introduces new hurdles. Nearly all organizations (95%) plan to increase Kubernetes usage in the next year, while simultaneously doubling down on AI and genAI capabilities. 96% of organizations say it’s important for them to provide efficient methods for the development and deployment of AI apps and 94% say the same for generative AI apps. This threatens to overwhelm platform teams even more if they don’t have the right tools and strategies in place.

As a result, organizations increasingly seek capabilities for GPU virtualization and sharing across AI workloads to improve utilization and reduce costs. The ability to automatically allocate AI workloads to appropriate GPU resources based on cost and performance considerations has become essential for managing these advanced technologies effectively.

Prioritizing automation and self-service

Our research reveals a clear mandate: Organizations must fundamentally transform how they approach infrastructure management to becoming enablers of self-service capabilities. According to our research, organizations are prioritizing proactive, automation-driven solutions such as automated cluster provisioning, standardized and automated infrastructure, and self-service experiences as top initiatives for developers.

Organizations are zeroing in on a range of cost management initiatives for platform teams over the next year, including:

Reducing and optimizing costs associated with Kubernetes infrastructure,

Visibility and showback into cloud and Kubernetes costs,

Providing chargeback to internal groups (finops).

The push toward automation and self-service represents more than just a technical evolution—it’s a fundamental shift in how organizations approach infrastructure management. Self-service automation allows developers to move quickly while maintaining guardrails for resource usage and cost control. At the same time, standardized infrastructure and automated provisioning help ensure consistent deployment practices across increasingly complex environments. The result is a more sustainable approach to platform engineering that can scale with organizational needs while keeping costs in check.

By investing in automation and self-service capabilities now, organizations can position their platform teams to handle future challenges more effectively, whether they come from new technologies, changing business needs, or evolving infrastructure requirements.

Empowering platform teams

Platform team challenges—from Kubernetes and multi-cloud management to generative AI implementation—are significant, but not insurmountable. Organizations that successfully navigate this landscape understand that empowering platform teams requires more than just acknowledging their importance. It highlights the need for robust, versatile tools and processes that enable effective cost management and standardization. Platform teams need comprehensive solutions that balance innovation with cost control, while optimizing resources efficiently without impeding modernization efforts. Empowered platform teams will be the key differentiator between organizations that survive and those that excel as the landscape continues to evolve with new challenges in cloud, Kubernetes, and AI.

Haseeb Budhani is co-founder and CEO of Rafay Systems.

—

New Tech Forum provides a venue for technology leaders—including vendors and other outside contributors—to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to doug_dineley@foundryco.com.