Gimlet Labs and Cerebras team up for 3,000-token-per-second inference

Gimlet Cloud will integrate Cerebras' wafer-scale silicon with GPUs to target ultrafast inference for agentic and real-time AI workloads.

By The Datatech Times Editorial Team AI level 2 of 5
AI level 2 of 5: drafted by our AI editorial assistant from the company's announcement; fact-checked, edited and signed off by Mark Walker, Editorial Director. What the levels mean
A long, brightly lit data center aisle features rows of server racks with active green and blue indicator lights extending into the distance.

Gimlet Labs and Cerebras Systems (NASDAQ: CBRS) have announced a collaboration to bring high-speed AI inference to production workloads through Gimlet Cloud, with a target throughput of up to 3,000 tokens per second. The first Cerebras-powered Gimlet Cloud data centre is expected to come online before the end of 2026, with joint customer engagements already underway and tokens being served in private deployments.

The combined solution pairs Cerebras' Wafer Scale Engine with conventional GPUs inside a disaggregated inference architecture. Rather than routing all inference traffic through a single class of accelerator, Gimlet's orchestration layer assigns each phase of model execution to the silicon best suited to it: the Cerebras chip handles latency-sensitive prefill stages, while GPUs absorb high-throughput decode work. The announcement also confirms that Gimlet will act as a launch partner for the CS-4, Cerebras' latest generation processor.

The deal

Zain Asgar, co-founder and chief executive of Gimlet Labs, said the partnership was designed to unlock new categories of real-time application. "By combining Gimlet's multi-silicon software with the Cerebras Wafer Scale Engine, we can run each phase of inference on the hardware best suited to it and plan to deliver up to 3,000 tokens per second at production scale," he said.

Sean Lie, co-founder and chief technology officer at Cerebras, framed the arrangement in economic terms, arguing that pairing the fastest tokens from Cerebras with GPU throughput improves data centre unit economics. Cerebras is backed by a stated roster of major customers including OpenAI, Group 42 and AWS, providing a degree of commercial validation for its inference claims, though the company's forward-looking statements note material risks tied to customer concentration and an early-stage cloud business.

The collaboration is expected to deepen over time to cover software integration, API design, developer tooling, optimisation and production operations.

Market context

Managed inference has become one of the most fiercely contested layers of the AI infrastructure stack. Hyperscalers (AWS Bedrock, Azure AI Foundry, Google Vertex AI) compete against GPU-cloud specialists and a growing number of inference-optimised startups, each attempting to differentiate on latency, throughput, cost per token or developer experience.

The specific angle Gimlet and Cerebras are pursuing, disaggregated inference across heterogeneous silicon, sits at the frontier of production deployment practice. Most enterprise workloads today still run on homogeneous GPU clusters, but agentic pipelines and voice or video AI are increasingly exposing the latency ceiling of that approach. Cerebras' wafer-scale design, which integrates a full processor onto a single silicon wafer rather than a chiplet array, gives it an architectural advantage in memory bandwidth and on-chip interconnect, which translates directly to lower time-to-first-token. Whether that advantage scales economically at volume remains the central question for buyers evaluating the platform.

Gimlet Labs is backed by Andreessen Horowitz and Menlo Ventures, both of which have been active in AI infrastructure investment over the past two years. The company describes its research programme as spanning automated GPU kernel generation, workload orchestration and heterogeneous hardware execution, positioning it in a relatively thin field of inference-focused software vendors rather than the broader GPU-cloud market.

Standards and regulatory read-across

Inference infrastructure increasingly intersects with data-residency and AI governance requirements. The EU AI Act's general-purpose AI obligations, phasing in through 2026 and 2027, require transparency and documentation for high-capability models served at scale. Enterprise buyers in regulated industries will look for data-centre location commitments and audit trails before committing production workloads. Neither Gimlet nor Cerebras disclosed which geographies the new data centre will serve or what compliance certifications (SOC 2, ISO 27001, FedRAMP) they are targeting in this release.

The 3,000-token-per-second figure is a forward-looking target, not a published benchmark result, and should be read accordingly until independent validation is available.

Sector

Topic

Geography