OpenAI and Broadcom unveiled Jalapeño this week — OpenAI’s first custom inference ASIC, built on TSMC N3P, taped out in roughly 16 months. SemiAnalysis flew to OpenAI’s labs to run independent benchmarks, and the number that matters isn’t a marketing claim: Jalapeño beats Nvidia’s Blackwell on performance-per-watt in nearly every workload they tested. That’s not a story about one company’s chip. It’s a preview of what your AI infrastructure bill looks like when your model vendor stops renting compute and starts owning the stack down to silicon.

The numbers, stated plainly

The current A0 silicon hits 13.4 PFLOPs of MXFP4 at 700W, paired with HBM4 at 15.4TB/s of memory bandwidth. In throughput terms: 700+ tokens/sec/user on DeepSeek R1, roughly 1,400 tokens/sec/user on GPT-OSS. OpenAI’s own benchmarks claim 1.5-1.9x more useful work per watt than Nvidia across GPT-OSS, DeepSeek R1, and Kimi K2.5 1T — and SemiAnalysis’s independent numbers back that up rather than debunk it, which is the part worth paying attention to. B0 stepping is already in fab, targeting roughly another 25% perf/watt improvement.

At rack scale: 128 Jalapeño ASICs per rack, up to 2,048 XPUs across a 16-rack scale-up domain, with a full host-plus-ASIC rack pulling about 160kW. That’s a real, deployable number, not a lab curiosity — this is being built for production inference at OpenAI’s scale, not as a research demo.

Why this matters more than another model release

Every model release changes what you can build. A custom inference chip from your model vendor changes what you pay to run it, and the two economics diverge in an important way.

Perf-per-watt is the real API pricing lever. Inference cost isn’t dominated by chip purchase price at hyperscaler volume — it’s dominated by power and cooling over the chip’s operating life. A vendor that’s 1.5-1.9x more efficient per watt than the merchant-silicon alternative has a structural cost advantage that shows up as either better margins or lower API prices, and OpenAI gets to choose which. Historically, when a frontier lab gets a cost advantage like this, some of it reaches customers as lower per-token pricing — because it’s also a competitive weapon against Anthropic and Google, both of whom are pursuing their own custom silicon paths (Google’s TPUs are already in their sixth-plus generation; Anthropic leans on both Trainium and Google’s TPUs).

It’s a signal about supply reliability, not just cost. 2025 and early 2026 were defined by GPU allocation being the actual bottleneck for a lot of teams — not model capability, but literally getting enough Nvidia hardware provisioned. A model vendor with its own ASIC pipeline, on its own fab allocation with TSMC, is buying itself independence from Nvidia’s allocation queue. If you build critical infrastructure on a vendor’s API, their supply chain resilience is now quietly part of your own uptime risk.

Nine-month design-to-tapeout is the detail that should worry chip incumbents, not you directly — but it changes your planning horizon. OpenAI used its own generative models to accelerate the hardware engineering process. If AI-accelerated chip design becomes a repeatable pattern across labs, the historical 3-4 year cadence for meaningful hardware generations compresses. That means your infra cost assumptions from 18 months ago are more likely to be stale, sooner, than they used to be.

Hands-on: what this changes in how you plan capacity

This isn’t hardware you’ll provision directly — it’s OpenAI’s internal infrastructure. But it changes three things you actually control:

# Before: cost model assumes flat $/token pricing with occasional step-changes
monthly_inference_cost = tokens_per_month * current_price_per_token

# After: build in a cost-curve assumption, not a flat rate,
# when custom-silicon vendors are in your provider mix
def projected_cost(tokens_per_month, current_price, months_out, annual_decline_pct=0.15):
    # Vendors with vertically-integrated silicon have historically
    # passed through efficiency gains faster than pure API resellers
    projected_price = current_price * ((1 - annual_decline_pct) ** (months_out / 12))
    return tokens_per_month * projected_price
  1. Re-run your build-vs-buy math with a steeper price-decline assumption for vertically-integrated vendors. If OpenAI’s cost-per-token structurally drops faster than a reseller passing through Nvidia pricing, that changes the multi-year math on self-hosting open-weight models purely for cost reasons — the API option may keep getting cheaper faster than your own infra does.
  2. Watch which workloads get Jalapeño-routed first. Vendors typically route their highest-volume, most latency-tolerant traffic to custom silicon first, since that’s where the ROI on the design investment shows up quickest. If your workload profile matches (batch-ish, not latency-critical), you may see price or availability improvements before latency-sensitive workloads do.
  3. Treat vendor supply-chain independence as a vendor-selection criterion, not just a curiosity. When you’re choosing between API providers for a critical path, “do they control their own inference hardware pipeline” is now a legitimate line item next to model quality and pricing — it’s a proxy for how insulated they are from the next GPU shortage.

The headline here isn’t “OpenAI built a fast chip.” It’s that model quality and infrastructure cost are starting to decouple from Nvidia’s roadmap for at least one major lab, and if that pattern holds across Anthropic, Google, and Meta too, the multi-year cost curve you’re planning against today is probably wrong.

Export for reading

Comments