Musk’s xAI unveils a new API service for Grok models

Elon Musk-backed generative AI venture, xAI, has unveiled a new API service for its Grok models to allow developers to underpin their applications and agents with xAI’s large language models (LLMs).  

Musk took to Twitter, now X, to announce that the API service was live and shared a link where developers could sign up for the service.

xAI API, currently in the public beta phase, is akin to OpenAI and Anthorpic’s API services that allow developers to call large language models such as GPT-4o, GPT-4o mini, Claude Opus, Sonnet, and Haiku.

The comparison with OpenAI and its likes assumes significance as one of the main reasons behind starting the venture last year in July was to compete with OpenAI and somehow match or surpass the competency and reliability of the GPT models.

In November, last year xAI unveiled its Grok-1 model, which though surpassed the performance of GPT-3.5, failed to beat GPT-4 — the then-latest model in OpenAI’s inventory.

API service to help underpin LLMs appears to be a path that most model providers and generative AI platform providers have adopted to allow developers to adopt their models.

For instance, Meta and Cohere already have API services. Hyperscalers, such as Google, Microsoft, AWS, and Oracle also provide options for various LLMs for specific use cases via VertexAI, Azure OpenAI, Microsoft Copilot Studio, Amazon Bedrock, and Oracle’s Generative AI service.  

Components of the API service and caveats

The xAI API service provides access to the xAI Console, which acts as the hub for the service for developers, according to the documentation listed on the company website.

Via the Console, developers can access xAI models such as the frontier Grok-2 and the lighter and more affordable Grok-2 mini. However, when we signed up for the service, a model named grok-beta was the only one available.

Capabilities of the Grok model, which the company describes as a “general purpose model,” include generating and understanding text, code, and images.

The xAI Console provides access to several tools that allow developers and their enterprises to create an API Key, invite a team member, set up billing, view and compare models for usage, track usage via the usage explorer, and a link to the API documentation.

Toby Pohlen, who is another founding member of xAI, took to X to suggest that the usage explorer is akin to similar explorers offered by major cloud providers.

The founding member also said that the company’s inference servers are designed to support multiple regions.

“We can run them on-premise or in the cloud (on any provider). This will be particularly helpful in the future when we want to bring low latency APIs to more regions,” Pohlen wrote on X.

However, the Console, in our case, showed us that the region was limited to us-east.

Additionally, Pohlen said that the entire backend system for the API was written in Rust.

xAI API pricing

Another point of difference between the xAI API and OpenAI API is the difference in pricing.

From the pricing information listed on the API portal, it seems that Grok-2 and Grok-2 mini are more expensive than OpenAI’s models such as GPT-4o and GPT-4o mini.

While the Grok-beta can process 131,072 tokens of context for both input and output, GPT-4o can process 1 million tokens. Additionally, Grok-beta costs $5 for its 131,072 input tokens versus GPT-4o’s $2.50 for a million input tokens.

On the output side, Grok-beta costs $15 for its limited token count versus GPT-4o’s $10 for 1 million tokens. Additionally, OpenAI also offers Batch API pricing which is at least 50% less than normal API requests.  

However, the latest o1 models provided by the Sam Atlman-led startup are more expensive than Grok-beta.

Other features of the API include the ability to integrate with OpenAI and Anthropic SDKs (software development kit) using JavaScript and Python.

Further, developers can integrate the xAI APIs into their application using REST, gRPC, or xAI’s Python SDK, the company wrote in its documentation.