Amazon Bedrock faces revamp pressure amid rising enterprise demands

Amazon’s flagship generative AI platform, Bedrock, could be in for a significant overhaul as customer expectations evolve and competition intensifies between the three major cloud providers — AWS, Microsoft, and Google Cloud.

Analysts suggest that growing customer demand for updated model offerings and tooling inside Bedrock may push AWS to rethink its strategy around its generative AI offerings, potentially signaling a major shift in its generative AI roadmap.

Enterprise customers have long voiced frustration over the absence of OpenAI’s popular LLMs and the lack of a cohesive agent-building framework within Bedrock, said Eric Miller, VP of data and AI at ClearScale, a cloud consultancy and AWS partner, about organizations it works with.

The problem around the unavailability of OpenAI’s LLMs is further compounded due to cost-related challenges that further complicate adoption, according to Derek Ashmore, AI enablement principal at Asperitas Consulting, a cloud consultancy firm and AWS partner.

“As enterprises must call OpenAI’s APIs separately to access its LLMs, they run into data egress fees when transferring results from OpenAI’s service back into AWS. There is also the challenge of additional integration overhead for managing external API calls outside of Bedrock’s unified environment,” Ashmore said.

Similarly, The Futurum Group’s lead of the CIO practice, Dion Hinchcliffe, pointed out that the act of calling external APIs introduces latency as well as security concerns.

“CIOs I talk with find this fragmentation increasingly untenable for production workloads,” Hinchcliffe said.

Amazon Bedrock’s problems with perception and tooling

Amazon Bedrock is grappling with a perception and tooling challenge that’s increasingly hard to ignore for enterprise customers.“Enterprises are increasingly looking for more than just raw access to foundation models — they want frameworks that simplify building applications. While Bedrock offers a strong selection of models, it lacks higher-level tools for agent orchestration, retrieval-augmented generation (RAG), and workflow automation — leading to a disjointed developer experience,” said Ashmore.

In contrast, Azure and Google Cloud have invested heavily in features like Copilot Studio, Azure AI Foundry, and Vertex AI Agent Builder, which provide these capabilities out of the box, Ashmore added.

The lack of tooling and the changing perception of Bedrock, according to Miller, is in the service’s origin story and how the market views it: “It has really set itself apart from rival offerings as more of a model marketplace or a nexus of models behind a common API interface rather than an orchestrating platform.” 

Late to the open-source agent development framework party?

Another challenge that AWS faces is in the open-source agent development framework category, whose usage has become a significant trend within the developer community. This is evident by the growing adoption of Crew AI, Hugging Face, Microsoft’s AutoGen, and Google’s Agent Development Kit (ADK).

While the cloud services provider did release Strands Agents, an open source AI Agents SDK last month, Google’s ADK and Microsoft’s AutoGen, which were released relatively earlier, have “already set a high bar for open source, developer-centric software for agent creation and is more widely adopted,” said Igor Beninca, data science manager at Indicium — a AI and data consultancy firm.

Additionally, Strands Agents, unlike ADK and AutoGen’s integration into Vertex AI and  Azure OpenAI and Azure AI Foundry, respectively, is not natively integrated into Bedrock, although it uses a few Bedrock tools — a growing demand among developers and CIOs, Hinchcliffe pointed out.

“Customers say that Strand Agents feel stranded as they are not integrated into Bedrock, forcing them to stitch workflows manually. This breaks AWS’ promise to customers that Bedrock is a managed, secure, and scalable generative AI service,” Hincliffe said.

However, AWS seems to be taking cognizance of customer and partner feedback on Bedrock.

The cloud services provider, according to Hinchcliffe, has been “quietly preparing a major uplift of Bedrock” focused on making the service agent-native, RAG-friendly, and finally elastic in pricing.

“Think: managed agent hosting, deeper memory or state support, and tunable orchestration across model endpoints. The goal is to move from ‘just inference’ to ‘intelligent automation,’ especially for large enterprises already deploying multi-agent systems via open source on raw EC2,” Hinchcliffe said.

The Information was the first to report the possible revamp of Bedrock.

Notably, AWS and its leadership have often said that its offerings are always based on and improved by customer feedback.

But is a major revamp really required?

Experts are divided on whether a major revamp of Bedrock is currently necessary, and the debate is based around enterprises voicing issues around trying to use Bedrock in conjunction with an open-source agent development framework, which rival offerings, such as Vertex AI, don’t pose.

“Currently, if customers want to use frameworks like AutoGen or CrewAI on AWS, they have to run them manually on EC2 instances — a bit of an anti-pattern for customers who seek to operate truly cloud native or desire to use a unified development environment,” Miller said.

However, Miller pointed out that customers are not, in all likelihood, asking for a major revamp but a new separate managed service that allows them to run open source agent development frameworks on AWS without managing their own EC2 infrastructure.

“They (customers) don’t want to size, deploy, manage, and maintain EC2 instances to run this software…they just want to turn something on inside the AWS Console, and start building via the console or SDK, and then pay for it consumption based as they go, while AWS manages the service uptime, etc,” Miller said.

Explaining further, Miller said that AWS is at a turning point in its generative AI strategy, comparing the potential revamp of Bedrock to the shift from Elastic Container Service (ECS) to Elastic Kubernetes Service (EKS) — when AWS adapted to customer demand for Kubernetes.

“If AWS launches a turnkey, cloud-native service for agentic frameworks today, it could be the EKS moment for Bedrock— delivering exactly what customers are asking for and setting a new standard in the space,” Miller added.

In contrast, Asperitas Consulting’s Ashmore said that “there is a need for AWS” to clarify its platform strategy as well as remain competitive with its rivals.

“Right now, the relationship between Strand Agents, Bedrock, Amazon Q, and SageMaker isn’t well-defined. A revamp could bring these together into a more cohesive, developer-friendly offering that simplifies how customers build and scale agent-based applications on AWS,” Ashmore added.

In addition, Ashmore pointed out that AWS may need to act fast as enterprises are already looking at alternatives that provide the capability to access an open-source agent development framework within a unified development platform.

For instance, ClearScale has built an offering — ClearScale GenAI workbench, which is a wrapper around Bedrock that adds agentic framework capabilities, specifically AutoGen, and many other features that make building agents with Bedrock relatively easy.