Copilot Studio Agents get a major upgrade via Microsoft 365 Copilot

Microsoft Ignite 2024 put an end to Microsoft’s dalliance with chatbots. Not because the company is walking away from natural language interfaces, but because it’s taking what it learned with its first Copilots and refocusing on tangible productivity gains from large language models (LLMs).

This is not surprising. Although LLM-powered chatbots are an amusing novelty with some useful text-generation features, Microsoft is delivering new ways of working with them, first grounding them in enterprise content with its Microsoft 365 Copilot and then adding tools for working with OpenAPI-powered services in its foundational AI orchestration tool, Semantic Kernel.

Much of Microsoft’s work during the past year or so has led to tools focused on what we’re calling “agentic AI.” Building on the 30-plus-year-old concept of autonomous agents, agentic AI mixes several different LLM-based development techniques to deliver self-assembling process automations that convert a natural language query into a long transaction that works across a set of well-defined service endpoints.

Build low-code agents with Copilot Studio

New tools such as Azure AI Foundry are driving some of the shift to agentic AI. Microsoft is putting the same tools in the hands of its low- and no-code developers in Copilot Studio. The intention is to kickstart agent development, not from traditional developers, but from business users who are at the heart of complex business processes and are looking for tools to simplify things.

Some of the Copilot Studio features detailed at Ignite 2024 had been announced at an event in London a month or so earlier. Ignite saw Microsoft put things in context, adding important details and showing how low-code development would integrate with more complex AI applications built with Azure AI Foundry.

Bringing custom and prepackaged AI together this way fits with the fusion teams model used to link Power Platform applications with the rest of your development process. It allows you to use Azure to build and tune custom multimodal AI models, which can then be quickly included in users’ own applications (with appropriate guardrails and API management rules).

Bringing Power Platform into the full developer ecosystem

Power Platform has stood apart from the rest of Microsoft’s stable of developer tools. The separation was practical; it sold to a different market and Microsoft wanted to avoid confusion. The logical replacement for tools like the classic Video Basic and Silverlight, it’s developed its own Excel Functions-derived programming language, with its connectors and actions the only link to the wider Microsoft ecosystem. That model is changing, and Copilot Studio’s rapidly developing agentic AI platform is leading the way.

Part of the change comes from Microsoft’s understanding that process automation requires significant input from the business alongside technology expertise. Agentic AI will require both programming and process-modeling skills to build out a pipeline of AI assistants supporting a long workflow. Mixing and matching AI agents in a pipeline lets you use the right models for the right piece of the workflow, for example, an OCR model feeds a tool that uses a Microsoft 365 Copilot call to schedule the appropriate field maintenance engineer for a specific component failure.

Using Copilot Studio with the Microsoft 365 Agent SDK

One of the more interesting new features announced at Ignite was a set of tools for building Microsoft 365-based Agents, allowing access to Copilot Studio applications from your C# code (with Python and Node.js support on the project road map). This new SDK bridges not only Azure AI Foundry and Copilot Studio but also lets you work with third-party AI platforms.

The Microsoft 365 Agent SDK has four key components. It offers multiple user interfaces, from the Microsoft 365 Copilot using tools like Copilot Pages, to Teams and even Slack. Grounding data comes from services that include the Microsoft Graph, Azure Fabric, and Azure AI Search. Everything is orchestrated from Semantic Kernel, with agent support from both Copilot Studio and Aure AI Foundry.

Integration between Copilot Studio and the Microsoft 365 Agent SDK is two-way: You can use Semantic Kernel to add memory and skills to a Copilot Agent, adding additional features, or your C# code can call an existing Copilot Agent without needing to build API integration thanks to the Power Platform’s library of connectors.

Using the two tools together extends your reach, bringing Copilot Agents to more than 15 channels, from chat and email to SMS and Teams. You’ll probably be familiar with most of them if you’ve used the original Bot Framework for building basic chatbots. The Microsoft 365 Agent Framework is its successor (in much the same way Copilot Studio builds on the original Power Virtual Agents service).

A flexible agent future

This mix of low code and pro code should give both platforms flexibility, allowing you to mix and match the features you need to build and deliver the appropriate agent-powered workflow for your specific business process needs. Where it used to be hard to go from a Power Platform prototype to a fully managed application, this approach will allow business teams to find problems and experiment with agentic workflow solutions, which can then be included in a larger business process automation project.

There’s more to building effective agents than code: You need good data sources as well as a variety of tunable models. The latest update to Copilot Studio uses the Azure AI Search vector indexing tool to provide data sources for tools using retrieval-augmented generation (RAG), with additional tools to support tuning responses to reduce the risks that come from LLMs generating incorrect responses. You will be able to use the Copilot Control System to provide appropriate policies for managing your services, including adding access controls to reduce the risk of sensitive data leaking through agent operations.

It will be interesting to see how tools like these step outside traditional workflows, having the next generation of AI models directly control user PCs and automating workflows that extend from the data center to desktops. Although Microsoft 365’s GraphQL APIs let today’s agentic AIs work with familiar productivity applications, there are still many enterprise applications that have bespoke user interfaces built using tools like Visual Basic. These are currently inaccessible from most AI tools.

What’s next? Agents on the desktop

It’s not hard to imagine using a desktop workflow automation tool like the bundled Power Automate to identify desktop application screens and controls that are necessary to complete a hybrid desktop/cloud workflow. Once you have them labeled, a next-generation LLM can respond to events in an agent workflow by controlling those applications, automating user interactions that previously would have added latency to a business process.

Applying agents to applications with no API beyond the Windows desktop is an intriguing prospect, especially if interactions can be carried out using Azure-hosted cloud PCs rather than interfering with a user’s normal tasks. Once you start to think about how agentic AI puts the robot into robotic process automation, services like Windows 365 start to make a lot more sense as part of a suite of tools that enable widescale deployment of these tools, even with applications written decades before we had access to the first LLMs.

Microsoft has chosen to focus much of its agentic AI tools on business process automation to manage long workflow-based transactions, with an underlying orchestration engine to manage context. This requires collaboration between software developers and users. By providing different ways to mix low-code tools with traditional development platforms, users have the ability to choose the approach that’s right for them, their experience with AI, and their business needs.

As always, it’s a pragmatic way of delivering a new technology. It can change quickly as new tools and models arrive, which they inevitably will—and faster than you expect.