Jalapeño: Inside OpenAI's First Custom AI Chip

For years, every conversation about the AI boom eventually circled back to the same company: Nvidia. If you wanted to train or serve a large language model at scale, you needed Nvidia's GPUs, and you got in line with everyone else. OpenAI has now taken its first real step toward changing that equation. On June 24, 2026, OpenAI and Broadcom pulled the wrapper off Jalapeño — OpenAI's first purpose-built AI chip, and the opening move in what both companies are calling a multi-generation compute platform.

This isn't a rebranded GPU or a science project. Jalapeño is real silicon, running real workloads, with real (if self-reported) benchmark numbers behind it. Here's what it is, how it was built, and why it matters.

What Exactly Is Jalapeño?

Jalapeño is an Application-Specific Integrated Circuit (ASIC) — a chip designed to do one job extremely well, rather than a general-purpose processor like a GPU that has to handle graphics, training, and inference all at once. OpenAI is calling it an "Intelligence Processor," and its one job is serving large language models to users: the token-by-token generation that happens every time someone sends a message to ChatGPT, runs a Codex task, or calls the OpenAI API.

That distinction matters. Nvidia's GPUs are marvels of general-purpose engineering, but that flexibility comes with overhead. OpenAI's pitch is that by designing a chip from a blank slate — with no legacy baggage from graphics or training workloads — it can strip that overhead out and build something that does inference, and only inference, closer to its theoretical hardware limit.

Engineering samples of Jalapeño are already running in OpenAI's labs at production-target frequency and power, processing live model workloads including GPT-5.3-Codex-Spark, while the company finishes qualifying the chip for large-scale deployment.

Why Build Your Own Chip At All?

Reading the tea leaves, OpenAI's chip ambitions go back further than most people realize. Reuters first reported the company exploring custom silicon back in 2023, and by late 2024 it had assembled an in-house hardware team — including engineers who had previously worked on Google's TPUs — to pursue an in-house design instead of the far more expensive route of building its own chip foundries. The Broadcom partnership was formally announced in October 2025, with OpenAI CEO Sam Altman describing computing commitments in the range of 10 gigawatts.

The underlying logic is simple economics:

• Training a model is a one-time (if enormous) upfront capital expenditure.
• Inference — actually serving that model to hundreds of millions of users, every day, forever — is a recurring operational cost that scales directly with usage.

OpenAI has said it now serves over 800 million weekly users. Shaving even a fraction of the cost and power off every single inference request compounds into billions of dollars in savings at that scale, and it gives OpenAI strategic leverage and supply diversification instead of total dependence on Nvidia and a handful of other chip vendors.

The Architecture: Solving the Prefill/Decode Problem

The most interesting part of Jalapeño isn't the marketing — it's the specific engineering problem it was built to solve. LLM inference happens in two very different phases, and they fight each other for hardware resources:

1. Prefill: The chip reads and processes the user's prompt. This is compute-intensive — it's essentially one massive parallel matrix multiplication problem.
2. Decode: The chip generates the response, one token at a time. This phase isn't limited by raw compute; it's limited by memory bandwidth, because the system has to repeatedly access the model's "working memory" (the KV cache) for every single token it produces.

A chip optimized for one phase tends to be inefficient at the other, and data has to keep moving between compute, memory, and networking resources — which introduces latency and leaves parts of the chip sitting idle while they wait for data to arrive.

OpenAI designed Jalapeño to minimize exactly that kind of data movement. The KV cache and other model state can be explicitly placed and kept physically close to the compute elements, and the chip's networking is treated as a first-class part of the architecture rather than an afterthought — large enough that an entire workload can stay inside one connected system instead of being split and shuttled between chips. Broadcom's Tomahawk networking silicon plays a role here, tying the compute chiplets together at the scale the workload demands.

Independent analysis by Tom's Hardware, based on photos of the chip package released by OpenAI, estimated the compute chiplet's die size at roughly 840 mm² — close to the physical reticle limit for current EUV lithography, and noticeably larger than most rival inference chips on the market. The package appears to be surrounded by six HBM3e memory modules, a choice that favors raw memory bandwidth over the cheaper DRAM used in many other inference accelerators. (OpenAI hasn't officially confirmed these exact physical specs, so treat them as an informed estimate rather than a spec sheet.)

Built in Nine Months — With AI Designing the AI Chip

Perhaps the most striking claim around Jalapeño is the timeline. Chip design cycles for advanced ASICs typically run 18–24 months. OpenAI and Broadcom say they went from initial design to manufacturing tape-out in about nine months — something they describe as the fastest ASIC development cycle ever achieved in high-performance semiconductors.

The stated reason is a mix of deep software-hardware co-development and, notably, using OpenAI's own AI models as part of the design process itself. According to OpenAI, earlier-generation models helped the team explore implementations and speed up design, measurement, and verification loops during the chip's development, while more recent models are now being used to help optimize and program the finished hardware. AI assistance was also applied to optimizing the chip's arithmetic circuits, letting engineers pack more compute into the design without blowing the schedule.

For select attention and mixture-of-experts components, AI-generated code reportedly ran 1.5 to 1.8 times faster than the equivalent hand-written implementations from human hardware experts.

That work didn't stop at tape-out. OpenAI says it deliberately designed Jalapeño to be a predictable target for AI-assisted programming — engineers describe workloads through local tensors and explicit communication patterns that both humans and AI systems can reason about. Using Codex together with an internal model called GPT-Astra, the hardware team says it brought three open-weight models that weren't part of Jalapeño's original launch plan up to high performance within about two months.

The Benchmark Numbers

Talk is cheap in the chip world, so it's worth looking at what OpenAI actually published. On August 25, 2026, the company released its first performance results, tested on InferenceX, a public benchmark suite from the analyst firm SemiAnalysis that measures the full, realistic process of serving an AI request — not just a synthetic best case.

OpenAI compared Jalapeño against Nvidia's current top-of-line rack systems (GB200 and GB300) across three publicly available large models, normalizing everything by power consumption rather than per-chip performance, since Jalapeño's 700W rating is well below the 1,200W–1,400W of the Nvidia systems it was tested against.

Model	Peak Throughput per kW	End-to-End Latency	Interactive-Mode Gain
GPT-OSS 120B	~1.9x higher	~1.7x lower	up to ~2.7x lower time-between-tokens
DeepSeek R1 670B	~1.7x higher	~3.6x lower	up to ~4.1x lower time-between-tokens
Kimi K2.5 (1T params)	~1.5x higher	~3.4x lower	up to ~3.8x lower time-between-tokens

Across all three models, OpenAI reported Jalapeño delivering 1.5x to 1.9x more useful AI work per watt at peak throughput, and 1.7x to 3.6x lower end-to-end latency, compared to the Nvidia systems tested. For highly interactive, low-latency use cases — the kind that matter most for responsive chatbots and multi-step AI agents — the reported advantage climbed to 2.1x–4.1x. OpenAI also noted that on its own internal frontier models (not publicly disclosed in detail), the gap widened further, suggesting the architecture pays off even more as workloads get larger and more demanding.

A caveat worth stating plainly: these are OpenAI's own published numbers, even if they were generated on a recognized third-party benchmark suite. Independent, hands-on verification by outside labs hasn't happened yet, so — as several industry outlets covering the announcement noted — the claims deserve a healthy dose of "wait and see."

Where Jalapeño Fits in the Bigger Picture

OpenAI is not abandoning Nvidia. The company has been explicit that it will continue deploying Nvidia and AMD hardware heavily for both training and inference — Jalapeño is additive, not a replacement. What it does give OpenAI is a lever: a first-party option that reduces dependence on any single vendor and lets the company optimize hardware and software together, something Nvidia's general-purpose GPUs can't be tailored to do for one customer's models specifically.

This also puts OpenAI in company with several other players who've already gone down the custom-silicon road: Google has its TPUs, Amazon has Trainium, Microsoft has its Maia chip, and Meta has MTIA. Reports have also surfaced of Anthropic pursuing its own custom inference silicon. Building your own chip has effectively become a rite of passage for any AI lab operating at frontier scale — the economics of inference at hundreds of millions of users simply demand it.

What's Next for OpenAI Silicon?

Jalapeño is planned for initial production deployment inside OpenAI's own infrastructure by the end of 2026, scaling up through 2029 in partnership with Broadcom, Celestica, and data-center operators including Microsoft. OpenAI has said Generation 2 is already deep in development, with Generation 3 taking shape behind it — meaning Jalapeño is explicitly a first step, not a finished product.

The Bottom Line

If the real-world numbers hold up anywhere close to what's been published, the practical effect for users will be faster ChatGPT responses, AI agents that can complete more steps without long waits, and API pricing that has more room to fall as OpenAI's cost of serving each token drops.

That's the actual point of a chip like this — it doesn't make the models smarter, but it makes the intelligence they already have faster and cheaper to actually deliver to hundreds of millions of people worldwide.