Here’s how Google is using LLMs for complex internal code migrations

Code migration is a critical process in maintaining software applications. It helps improve performance and resilience, keeps systems up to date, and removes stale or irrelevant code.

But it can be complex and time-consuming on a number of levels, not the least of which is the fact that the code is very often sprawled across a variety of environments. And, while AI has emerged to support a number of lower-level programming tasks, the technology has yet to be able to handle the convoluted, confusing job of code migration.

But now Google says it may have overcome that challenge, offering up a new step-by-step process and common toolkit in which large language models (LLMs) find files that need to be changed. Already, the tech giant says its process has accelerated migrations by 50%.

“This approach has the potential to radically change the way code is maintained in large enterprises,” a group of authors from Google Core and Google Ads wrote in a new “experience report” describing their approach. “Not only can it accelerate the work of engineers, but make possible efforts that were previously infeasible due to the huge investment needed.”

Ultimately, Google’s goal was to identify opportunities for LLMs to provide additional value and support scale without requiring difficult-to-maintain abstract syntax trees (AST). These are widely used to represent the structure of a program or snippet of code, but they are deterministic — that is, outcomes are already identified — and code migration use cases span much more complex constructs that are difficult to represent with ASTs, the authors explained.

“Achieving success in LLM-based code migration is not straightforward,” the authors noted. “The use of LLMs alone through simple prompting is not sufficient for anything but the simplest of migrations. Instead, as we found through our journeys, and as described in the case studies in this paper, a combination of AST-based techniques, heuristics, and LLMs are needed to achieve success. Moreover, rolling out the changes in a safe way to avoid costly regressions is also important.”

Google defined success as AI saving at least 50% of the time for the end-to-end work. This comprised the code rewrite, the identification of migration locations, reviews, and final rollout. That milestone was ultimately achieved, as reported by engineers. In the end, 80% of code modifications were fully AI-authored.

“Anecdotal remarks from developers suggest that even if the changes are not perfect, there is a lot of value in having an initial version of the changelist already created,” the report said.

Use case: Google Ads IDs

One of Google’s largest business units, Google Ads, is built on a code base of more than 500 million lines of code. It has dozens of numerical unique ID types that reference different resources (known as “handles”) such as users, merchants, and campaigns. These typically are defined as 32-bit integers in C++ and Java; however, they needed to be converted to 64-bit IDs to avoid scenarios where the value of the ID was too large to be accommodated using 32-bits, the report authors explained.

But there are numerous challenges to moving from 32-bit to 64-bit. Within Google, IDs are very generally defined and not easily searchable. This makes the process of finding them through static tooling “non-trivial.” Compounding this is the fact that Google Ads has tens of thousands of code locations, making tracking of changes extremely difficult.

This is one of the areas where Google applied its LLM-powered code migration process. In the first step, an engineer finds the IDs, file supersets, and locations they want to migrate. The changes are then generated in the LLM, creating a feedback loop of testing and iteration. The engineer finally reviews the LLM-generated code (as they would any other code), changing and correcting as needed. Then, the changes are then split (sharded) and sent off for final review by the owners of each part of the codebase.

Updating to JUnit4

In another example, a group of teams at Google had a “substantial” set of test files that still used the now outdated JUnit3 library, a unit testing open-source framework for Java. Updating them manually would be a big investment, and old tests can negatively impact the codebase, the Google authors explained.

“They are technical debt and tend to replicate themselves, as developers might inadvertently copy old code to produce new code,” they wrote.

Google developers used the LLM to update what they called a “critical mass” of JUnit3 tests to the new JUnit4 library. With this technique, they were able to migrate 5,359 files, modifying more than 149,000 lines of code in 3 months.

Cleaning up stale code

A further use case involved cleanup of experimental code that had gone stale. Again, this process is time-consuming, so AI was asked to perform the following steps:

Find code locations where flags (experiments) are referenced

Delete code references to the flag

Simplify any conditional expressions that depend on the flag

Clean any dead code

Update tests and delete useless tests

Techniques can be applied anywhere

The framework is not restricted to Google’s software. As the report authors note: “We believe the techniques described are not Google-specific, and we expect that they can be applied to any LLM-powered code migration at large enterprises. The toolkit is versatile and can be used for code migrations with varying requirements and outputs.”