How LinkedIn built an agentic AI platform

Basic chatbots get much of the publicity associated with modern AI platforms, but they have limited use cases, providing a simple natural language interface to search tools. It’s certainly useful, provided you implement fine-tuning and grounding in your own data, but it is still best thought of as an extension of existing search tools.

AI has other uses: embedding the technology inside enterprise IT stacks, providing advanced filtering and summarization, using it for translation and voice recognition, and simplifying interactions through natural language. But that means incorporating AI as part of our existing development stacks, formalizing how we implement AI solutions and treating them as just another tool that offers new capabilities that extend what we can do with software.

From chatbot to application components

Building AI technology into enterprise applications forces you to remain aware of both its capabilities and its limitations. Agentic AI allows us to treat AI as orchestrated APIs, with data sources provided by Model Context Protocol (MCP) servers and agent-to-agent authentication and authorization via the under-development Agent2Agent (A2A) protocol.

It’s interesting to see how early adopters have begun to formalize using AI tools in their development tool chains. Last November LinkedIn unveiled its approach to a generative AI application stack, and now the company is building, testing, and monitoring agentic AI applications with a focus on longer interactions and workflows.

One thing I’ve noticed over the past few years of writing about AI application development is that developers often repurpose older coding techniques in order to quickly build modern applications. The LinkedIn framework is clearly building on existing distributed application development methods, implementing agents as part of a messaging-based platform.

Working with familiar concepts

LinkedIn Distinguished Engineer Karthik Ramgopal agrees. “Even when we are building agents, the reality is you still have a user-facing application,” he says. “You still have a large-scale distributed system in the background. So rather than invent everything from the ground up, you lean in on familiar technologies but add the layers needed to make agents work at scale.”

This explains the tendency of agent-based applications to fall back on messaging architectures. Ramgopal points out, “The reason we and almost everyone else are falling back to messaging as the abstraction is because it’s incredibly powerful. You have the ability to communicate in natural language, which is, you know, pretty important. You have the ability to attach structured content.” The use of structured and semistructured information is becoming increasingly important for agents and for protocols like A2A, where much of the data is from line-of-business systems or, in the case of LinkedIn’s recruitment platform, stored in user profiles or easy-to-parse resumes.

The orchestrating service can assemble documents as needed from the contents of messages. At the same time, those messages give the application platform a conversation history that delivers a contextual memory that can help inform agents of user intent, for example, understanding that a request for available software engineers in San Francisco is similar to a following request that asks “now in London.”

Building an agent life-cycle service

At the heart of LinkedIn’s agentic AI platform is an “agent life-cycle service.” This is a stateless service that coordinates agents, data sources, and applications. With state and context held outside this service in conversational and experiential memory stores, LinkedIn can quickly horizontally scale its platform, managing compute and storage like any other cloud-native distributed application. The agent life-cycle service also controls interactions with the messaging service, managing traffic and ensuring that messages aren’t dropped.

Agents are built using familiar gRPC interfaces, with application developers using libraries that handle conversions between messaging and API calls. Usefully, agent developers can have gRPC proto 3 options hold the agent metadata for use by the agent life-cycle service.

One key feature of the agent life-cycle service is support for long-running tasks and for managing the shift between interactive and batch operations. This brings AI-based applications closer to the model of contextual computing that Microsoft has been trying to deliver for more than 30 years, where agent memory functions give the AI agent apps a way to manage user intent, user context, and agent context.

Keeping humans in the loop

As well as context, the agent life-cycle service is intended to manage authorization and authentication and support role-based authentication. This is important. In much of the world, the personal data stored in the LinkedIn platform is regulated and needs to be protected to ensure privacy. This also requires a human in the loop, making decisions and guiding operations. For example, if an application is used to communicate via email, all generated emails should be editable and need to be explicitly sent by the user.

With much of LinkedIn’s platform focused on its role as a recruitment tool, the first public application built using its new framework is an update to its Hiring Assistant service. Developing tools that affect people’s lives has helped define key principles that aim to avoid a pure algorithmic approach, keeping humans involved to ensure the system stays trustworthy and safe. The intent is not to replace humans but to provide an aid that can help them work more effectively. The underlying agents need to collaborate with users to clarify decisions, get feedback, and provide inputs that are stored in contextual memory and can refine future actions.

The resulting application uses natural language interfaces to quickly filter job candidates based on recruiter queries within the existing LinkedIn platform, displaying matches and helping automate the hiring process. There are conversational elements, but the tool is embedded in familiar screens. Like many of the best AI tools, it’s designed to save time and help with existing processes, not automate them completely.

Using observability to manage agents

Along with an architecture for managing and orchestrating agents, the LinkedIn agentic technology stack provides a framework for including observability features, building on familiar technologies such as OpenTelemetry to instrument much of its operation. This allows you to see how and when agents call services and what data they use. You can use observability data to understand how an application is working, for both debugging and reliability purposes, as well as for providing necessary information required to ensure regulatory and privacy compliance. As Ramgopal notes, “You need to have the same rigor as you have for traditional software systems in terms of observability and monitoring, knowing what’s going on and auditing.”

It’s important to understand that this is key to LinkedIn’s approach to AI, that it’s another part of its existing enterprise architecture. Ramgopal describes it as part of the transition from pilot projects to everyday tools. “I think a lot of this gets lost in a bunch of demos people do of simple applications—hey, my LLM can do something fancy—but when you actually roll out an enterprise system at scale, all these things actually come into the picture and you have to think about them.”

AI in distributed architectures

One important aspect of LinkedIn’s internal tool is that it enables developers to understand that interactions can be non-deterministic. In the development playground, you can see the consistency trade-offs that come from using a distributed agent platform and decide how to balance latency and consistency. It’s reminiscent of the ways Microsoft exposed consistency models in its Cosmos DB platform, helping you choose the approach that works best for your code.

There are other trade-offs with a platform like this, such as the availability and cost of GPUs for at-scale inferencing. With the first application built on LinkedIn’s platform being designed to help recruiters find candidates for a role, much of the overall assessment process can be delivered asynchronously, treating agent operations as a batch process to save resources. However, agentic AI allows the platform to be flexible, for example, showing possible high-priority job candidates as soon as they express an interest in looking for a new role and informing recruiters as quickly as possible.

Treating agentic AI as a distributed application is key to this approach; agents are modules orchestrated by the core AI application, which controls the overall workflow, treating it as a life cycle that manages connections to agents and data sources.

LinkedIn’s framework shows that modern AI isn’t some special thing that needs to be treated differently from the rest of the code we build. Instead, it’s best to think of it as simply another piece in a cloud-native distributed architecture that’s part of our existing observability framework. Then we can start finding more uses for AI than yet another chatbot.