AWS offers new service to make AI models better at work

Enterprises are no longer asking whether they should adopt AI; rather, they want to know why the AI they have already deployed still can’t reason as their business requires it to.

Those AI systems are often missing an enterprise’s specific business context, because they are trained on generic, public data, and it’s expensive and time-consuming to fine-tune or retrain them on proprietary data, if that’s even possible.

Microsoft’s approach, unveiled at Ignite last month, is to wrap AI applications and agents with business context and semantic intelligence in its Fabric IQ and Work IQ offerings.

AWS is taking a different route, inviting enterprises to build their business context directly into the models that will run their applications and agents, as its CEO Matt Garman explained in his opening keynote at the company’s re:Invent show this week.

Third-party models don’t have access to proprietary data, he said, and building models with that data from scratch is impractical, while adding it to an existing model through retrieval augmented generation (RAG), vector search, or fine-tuning has limitations.

But, he asked, “What if you could integrate your data at the right time during the training of a frontier model and then create a proprietary model that was just for you?”

AWS’s answer to that is Nova Forge, a new service that enterprises can use to customize a foundation large language model (LLM) to their business context by blending their proprietary business data with AWS-curated training data. That way, the model can internalize their business logic rather than having to reference it externally again and again for inferencing.

Analysts agreed with Garman’s assessment of the limitations in existing methods that Nova Forge aims to circumvent.

“Prompt engineering, RAG, and even standard supervised fine-tuning are powerful, but they sit on top of a fully trained model and are inherently constrained. Enterprises come up against context windows, latency, orchestration complexity. It’s a lot of work, and prone to error, to continuously ‘bolt on’ domain expertise,” said Stephanie Walter, practice leader of AI stack at HyperFRAME Research.

In contrast, said ISG’s executive director of software research, David Menninger, Nova Forge’s approach can simplify things: “If the LLM can be modified to incorporate the relevant information, it makes the inference process much easier to manage and maintain.”

Who owns what

HFS Research’s associate practice leader Akshat Tyagi, broke down the two companies’ strategies: “Microsoft wants to own the AI experience. AWS wants to own the AI factory. Microsoft is packaging intelligence inside its ecosystem. AWS is handing you the tools to create your own intelligence and run it privately,” he said.

While Microsoft’s IQ message essentially argues that enterprises don’t need sprawling frontier models and can work with compact, business-aware models that stay securely within their tenant and boost productivity, AWS is effectively asking enterprises not to settle for tweaking an existing model but use its tools to create a near–frontier-grade model tailored to their business, Tyagi said.

The subtext is clear, he said: AWS knows it’s unlikely to dominate the assistant or productivity layer, so it’s doubling down on its core strengths of deep infrastructure, while Microsoft is playing the opposite game.

Nova Forge is a clear infrastructure play, Walter said. “It gives AWS a way to drive Trainium, Bedrock, and SageMaker as a unified frontier-model platform while offering enterprises a less expensive path than bespoke AI labs.”

The approach AWS is taking with Nova Forge will curry favor with enterprises working on use cases that require precision and nuance, including drug discovery, healthcare, industrial control, highly regulated financial workflows, and enterprise-wide code assistants, she said.

Custom LLM training costs

In his keynote, Garman said that Nova Forge eliminates the prohibitive cost, time, and engineering drag of designing and training a LLM from scratch — the same barrier that has stopped most enterprises, and even rivals such as Microsoft, from attempting to provide a solution at this layer.

It does so by offering a pre-trained model and various training checkpoints or snapshots of the model to jumpstart the custom model building activity instead of having to pre-train it from scratch or retrain it for context again and again, which AWS argues is a billion-dollar affair.

By choosing whether they want to start from a checkpoint in early pre-training, mid-training, or post‑training, said Robert Kramer, principal analyst at Moor Strategy and Insights, “Enterprise choose how deeply they want their domain to shape the model.”

AWS plans to offer the service through a subscription model rather than an open-ended compute consumption model. It didn’t disclose the price publicly, referring customers to an online dashboard, but CNBC reported that Nova Forge’s price starts at $100,000 per year.

Enterprises can start building a custom building a model via the new service on SageMaker Studio and later export it to Bedrock for consumption, AWS said. Nova Forge’s availability is currently limited to the US East region in Northern Virginia.

This article first appeared on CIO.