Private cloud still matters—but it doesn’t matter most

For all the hype about “cloud-first” strategies, the enterprise data center is far from extinct. In fact, roughly half of enterprise workloads still run outside the public cloud, residing in on-premises data centers or private clouds. Industry surveys reinforce this point: According to Forrester, 79% of large enterprises have implemented internal private clouds. In other words, the private cloud remains a first-class citizen in enterprise IT. Years into the public cloud revolution, we’ve arrived at what VMware’s Michael Coté calls a “50:50 equilibrium” between on-premises and public cloud deployments.

Why do so many organizations stick with private cloud? There are plenty of sensible reasons, and they generally boil down to business realities rather than Luddite cloud resistance. Companies have poured capital into their own data centers and “private cloud” platforms (often powered by virtualization and container stacks), and many workloads hum along just fine there.

Still, those workloads are not the kind of things that will set your company apart or turn you into The Next Big Thing. Public cloud has been growing much faster than private cloud for a long time and for good reason. Let’s talk about why.

Change is hard

In my interviews with IT leaders, they generally mention a few reasons for a dogged persistence with private cloud:

Data gravity and locality: Moving to or replicating massive data sets in a public cloud can be impractical or costly. Keeping those workloads on premises avoids data transfer headaches and expenses.

Ecosystem integration: Many on-premises apps are tightly integrated with ERP systems, legacy services, and low-latency networks. Keeping interdependent systems together in a private environment can improve performance and reliability.

Governance and control: Private clouds offer perceived security and compliance benefits. (Often that safety is more perception than reality, but here we are). Enterprises with strict data governance or regulatory requirements often feel more comfortable keeping sensitive workloads within their own controlled infrastructure.

Familiarity and cost predictability: Organizations have deep expertise in managing their existing environments. For steady-state workloads, owning hardware offers more predictable costs. Enhancing an existing private setup with developer-friendly platforms can deliver flexibility without the uncertainty of public cloud billing spikes.

In short, the enduring appeal of private cloud is that it’s a known quantity. It’s the server infrastructure you control, tune, and largely “set and forget” for predictable demand. Even as public cloud usage has skyrocketed, enterprises have found a comfortable balance: Keep stable, well-understood systems on a private footing while using public cloud for most everything else. It’s no surprise then that analysts see hybrid strategies dominating. IDC forecasts global spending on private clouds (including hosted private infrastructure) to reach about $66 billion by 2027, a hefty sum, even if it pales next to the $815 billion that poured into public cloud in 2024.

Elastic is the new normal

Yet if the private cloud represents the comfortable, status-quo side of enterprise IT, the public cloud is increasingly the experimental, future-facing side. The equilibrium that Coté notes is real, but it masks an important dynamic: Most of the new, industry-defining innovation is happening on public cloud infrastructure. This has been a trend for a long time. Matt Wood, former data science chief at AWS, once put the reason to me this way:

The companies that go out and buy expensive infrastructure find that the problem scope and domain shift really quickly. By the time they get around to answering the original question, the business has moved on. You need an environment that is flexible and allows you to quickly respond to changing big data requirements.

We saw this vividly in the past two years as generative AI went from gimmick to boardroom priority; enterprises needed the agility to scale up or down genAI experiments. Practically overnight, companies hungry to build or leverage AI capabilities scrambled for scalable infrastructure—and found it in abundance at the big cloud providers.

The numbers tell the story. Cloud adoption keeps climbing, with more than half of enterprise workloads now running in public clouds (a “tipping point” per Flexera’s latest State of the Cloud report). Despite chatter about cloud repatriation (moving workloads back on premises), actual reversals have been minimal; only about 21% of cloud workloads have been pulled back, far outweighed by new cloud migration and growth. In fact, 2023 did not turn into the year of mass cloud retreat—quite the opposite. As InfoWorld’s own cloud columnist David Linthicum predicted, when it comes to advanced IT services such as AI and analytics, public clouds typically prove more economical and practical. Sure enough, when cost-conscious executives tapped the brakes on generic cloud spending last year, one area kept surging: AI workloads in the cloud.

All three of the hyperscalers (Amazon Web Services, Microsoft Azure, and Google Cloud) reported accelerating cloud revenue growth tied to AI demand. By late 2024, 79% of organizations were using or experimenting with artificial intelligence and machine learning services from public cloud providers. Some 72% of companies report at least some use of generative AI, often delivered via cloud APIs or platforms. This wave of enterprise AI experimentation isn’t happening in private data centers, not for the most part. It’s happening in cloud data centers and at the network edge.

AI: Where elasticity hits overdrive

AI is particularly hungry for the cloud’s elasticity. Training a new machine learning model or scaling up a generative AI application isn’t a steady nine-to-five job. It’s a bursty, spiky, resource-intensive endeavor. One week you’re fine-tuning a large language model and need dozens of Nvidia GPUs for a few days; the next week those GPUs sit idle. Public clouds shine here by letting companies rent massive compute power on demand and then turn it off when the job’s done. This flexibility isn’t just a nice-to-have feature, it’s essential for modern AI development. Unlike legacy enterprise apps that can sit on stable servers for years, AI workloads are inherently cloud-native.

Small wonder that the cloud hyperscalers are investing heavily to try to keep up with demand. So is Cloudflare, more known for its network than its cloud, which is rolling out Nvidia GPUs in more than 100 cities worldwide, letting developers run AI inference right where users are.

Such offerings underscore an industry truth: The frontier of enterprise computing has expanded beyond the walls of the private data center. It now lives in the highly elastic, geographically dispersed infrastructure of public clouds and cloudlike networks. These are environments where capacity can scale up by orders of magnitude on demand, where new services (from serverless functions to AI APIs) launch weekly, and where economies of scale drive down the cost of experimentation.

Private cloud’s future

None of this is to suggest that private clouds are obsolete—far from it. Large enterprises will maintain significant on-premises footprints for the foreseeable future, for all the reasons we’ve discussed. The enterprise IT landscape in 2025 is undeniably hybrid and likely always will be. But it’s equally undeniable that the center of gravity for innovation has shifted. When a new opportunity emerges—say, deploying a breakthrough AI model or scaling a customer-facing app to millions of users overnight—companies aren’t spinning up a new on-premises cluster to meet the moment. They’re tapping the virtually unlimited resources of AWS, Azure, Google, or edge networks like Cloudflare. They’re doing so because cloud offers experimentation without hardware procurement, and success isn’t gated by how many servers you happen to own.

Private clouds excel at running the known and steady. Public clouds excel at unleashing the unknown and extraordinary. As we reach a cloud/on-prem equilibrium, this division of labor is becoming clearer. The day-to-day workloads that keep the business running may happily live in a familiar private cloud enclave. But the industry-defining projects, the ones leaders hope will define the business’s future, gravitate to infrastructure that can stretch to any size, in any region, at a moment’s notice. In the cloud era, elasticity is the mother of invention, and that’s one mother most on-premises setups simply lack.