The Pulse
Together AI Expands IBM Cloud Inference With NVIDIA B300 GPUs
Together AI says it is the first customer running on a dedicated, large-scale NVIDIA B300 inference cluster on IBM Cloud, which also uses NVIDIA Spectrum-X networking. IBM’s August announcement described a $240 million, multi-year agreement
AI.info Team · Written with AI ·
Together AI says it serves hundreds of trillions of tokens each month—and is now running on a dedicated NVIDIA B300 inference cluster on IBM Cloud. The company’s October 6 update describes the deployment as the first dedicated, large-scale inference cluster of its kind on IBM Cloud, with Together AI as its first customer. IBM supplies the cloud infrastructure, NVIDIA the GPUs and networking, and Together AI the inference platform.
A $240 million deal behind the B300 cluster
The update follows an August 11 announcement by IBM of a multi-year, $240 million agreement with Together AI. IBM said it planned to deploy a large cluster of NVIDIA HGX B300 systems on IBM Cloud, with availability expected in the first quarter of 2027. Together AI would use the systems to serve open-source models.
That earlier timetable sits uneasily beside the October post’s statement that Together AI is already “the first customer running on it.” The new post gives no deployment date or explanation of how that status relates to IBM’s Q1 2027 projection. It also does not say how many B300 systems the cluster contains. IBM’s release describes the expected availability as a projection, and notes that statements about its future direction and intent may change or be withdrawn.
Inference capacity for open models
The cluster is built for inference: running models to generate responses after training, rather than training models from scratch. Together AI says it operates the inference layer, while IBM provides the cloud and NVIDIA supplies B300 GPUs and Spectrum-X Ethernet networking. The companies frame the arrangement as a way to serve production workloads using open models.
Together AI’s October post describes monthly demand as hundreds of trillions of tokens across more than a million developers. IBM’s August announcement put the monthly figure at 400 trillion tokens, a specific company-reported measure of usage. Neither announcement identifies that figure as a capacity target for the new cluster.
IBM said the agreement is intended to help Together AI provide open-model inference to enterprises, and described the cluster as designed for fast, efficient production workloads. The release also said NVIDIA’s comparison of the system with prior generations was 30 times more AI-factory output. That is a claim attributed to NVIDIA, not a performance benchmark for Together AI’s deployment.
IBM’s announcement placed the deal within Together AI’s wider business: it reported an $800 million Series C financing round at an $8.3 billion valuation and described the company’s platform as covering inference, training, fine-tuning and agentic workflows. IBM said Together AI selected IBM and NVIDIA for their product roadmaps and ability to deliver GPU capacity at the pace it required.
What the partners say—and what remains unspecified
Together AI chief executive Vipul Ved Prakash described the intended role of the partnership in IBM’s August announcement:
“Enterprises want the performance of the best frontier models without the closed-model price tag, and that only works if the infrastructure underneath is fast and reliable at scale,” said Vipul Ved Prakash, CEO at Together AI. “Working alongside IBM with NVIDIA gives us that foundation. This cluster lets us bring production-grade inference to more companies, faster, and it's a big step in our push to make open-source AI the obvious choice for enterprises.”
The October post does not name customers beyond Together AI or provide pricing, performance benchmarks, GPU count, or customer access dates. It also leaves the gap between the August availability forecast and its claim of a customer already running on the cluster unexplained. The announcements identify the partners, hardware and intended use, but not the cluster’s size or when other customers can use it.