Are we worse at cloud computing than 10 years ago?

Is it just me, or are we seeing more cloud project failures today than 10 years ago? Logic suggests we improve over time, but the metrics don’t support that assumption.

A cloud project 10 years ago typically involved migrating a few test programs and systems. Now, the systems involved are much more complex, with many more moving parts that affect multiple or all aspects of an enterprise’s operations. Today’s push toward AI means that complicated, data-intensive systems are now the preferred models for cloud systems. Due to the skills shortage and planning problems, these complex systems present significant obstacles to enterprise cloud adoption even on a good day.

We need to call in the A-Team to get cloud and AI projects done on time, done on budget, and done right. Unfortunately, the A-Team has a years-long waiting list. There are just not enough cloud migration and development skills to go around. Many organizations are settling for “less than ideal” talent who make incorrect calls and put cloud and AI projects on the path to failure.

Cloud migration projects fail or stall

Tech research giant Gartner states that 83% of all data migration projects fail and that more than 50% of migrations exceed their budget. Consider these additional statistics from SoftJourn: 50% of cloud migration projects either fail or stall, 56% of businesses encounter compliance and security challenges, and 44% of companies initiate their cloud migration with insufficient planning. This is not good news for anyone.

Let’s explore the key factors that contribute to this dismal success rate:

Inadequate planning. A lack of preparation leads to compatibility issues, unexpected costs, and technical roadblocks that could have been anticipated with proper assessment. Cloud projects have many dependencies. You need to pick a database before picking a development platform, and you have to determine performance requirements before moving to containers and microservices. I’m seeing more projects stall or fail due to the lack of simple planning.

The increasing complexity of IT systems. Organizations struggle with intricate IT architectures and interdependencies. I’ve covered the complexity issue to death because complexity is becoming the “silent killer” of cloud development and deployment. It can be managed, but it requires adequate planning (see the previous point).

The talent gap. The shortage of technical cloud expertise makes it increasingly difficult for organizations to execute and maintain cloud initiatives. This has become a critical bottleneck in cloud project success.

Uncontrolled cloud costs. Many organizations are seeing unexpected increases in post-migration operational expenses. The lack of adequate cost controls and automated mitigations leads to budget overruns and project failures. Projects coming in on budget are rare. Also, as I covered in my latest book, enterprises spend about 2.5 times the amount they budgeted to operate their cloud-based systems. Although finops can address some of these issues, strategic cost-planning problems are not being effectively managed.

Compliance and security challenges. Compared to earlier cloud adoption phases, today’s projects face a significant increase in security-related complications and compliance issues.

Post-migration application performance. The fact that many organizations struggle with application performance indicates that we need to be more effective at maintaining service levels during cloud transitions.

Keys to cloud project success

Rapid advancements in cloud technologies combined with mounting pressures for digital transformation have led organizations to hastily adopt cloud solutions without establishing the necessary foundations for success. This is especially common if companies migrate to infrastructure as a service without adequate modernization, which can increase costs and technical debt.

The growing pressure to adopt AI and generative AI technologies further complicates the situation and adds another layer of complexity. Organizations are caught between the need to move quickly and the requirement for careful, strategic implementation.

This decline in success rates is a critical warning sign for the industry. Our current approaches to cloud computing projects need serious reconsideration and improvement. The good news is there are ways to fix the problems. Focus on these key areas to address the most pressing cloud computing challenges:

Comprehensive planning. Include thorough application assessment, dependency mapping, and detailed modeling of the total cost of ownership before migration begins. Success metrics must be clearly defined from the outset.

A phased approach. Start with less critical applications and smaller projects to build expertise before scaling more challenging ones.

Skills development. Build internal cloud centers of excellence with targeted training programs and strategic partnerships with managed service providers.

Strong governance. Continuous monitoring and optimization processes must become standard practice, along with explicit operating models, robust cost management frameworks, and comprehensive security guidelines.

Modernize applications. When it comes to modernization, organizations must consider the appropriate refactoring and cloud-native development based on business value rather than novelty.

The overarching goal is to approach cloud adoption as a strategic transformation. We must stop looking at this as a migration from one type of technology to another. Cloud computing and AI will work best when business objectives drive technology decisions rather than the other way around.

Lessons from the past

This isn’t the first or last time we will face a crisis in IT. I’m old enough to remember when PCs first appeared in corporate offices. More than 90% of employees (including those in IT) had no idea what to do with them. To compound the PC problems, cell phones showed up in corporate offices soon after. The skills to seamlessly integrate these systems into enterprise operations didn’t exist. Most organizations stumbled around in the dark for much longer than they should have because they couldn’t find or train or afford the talent they needed to make things right. The typical rallying cry became, “Good luck, everyone!”

Cloud and AI are today’s iterations of PCs and cell phones. We will survive this crisis, but let’s learn from some of our past mistakes. If you can’t find the talent, be willing to invest in training. Put away the darts and dart boards. Build comprehensive short- and long-term planning into the budget for all aspects of your cloud and AI projects. Pursue projects that your current staff can handle. Study what’s broken and devise a viable plan to fix it. By fixing what’s broken, your staff should learn how to do things right the first time.

Easy, right? Good luck, everyone.