Should You Switch to Open-weight Models?
Open-weight models are making headlines this week. In this post, we breakdown what you’re actually buying when you pay for a token and how to think about the switch decision for your own product.
Earlier this year, a research team at Bridgewater (the hedge fund) took an open model, tuned it on their own financial data, and ended up with something small and cheap that scored 84.7% on financial reasoning benchmarks. That beat the leading proprietary models. At less than a tenth of the cost.
If you’re a founder or a PM you’d read that story as “open models are good enough now, switch and save money,” and that reading could be wrong in a way.
In this post, we’ll break down what you’re actually buying when you pay for a token, why open weights only remove one part of that bill, who is benefiting with everyone switching, and how to think about the switch decision for your own product. We’ll use one simple example throughout:
Let’s say you’re the PM for a feature that resolves billing disputes. A customer says they were charged twice, and your agent reads the account, checks the payment history against your refund policy, and drafts a response with a number in it. High volume, token-hungry, and the scenario where being wrong could cost actual money.
By the end of this post, you should be able to answer, for your specific feature/ product, whether open-weight models make sense or not and we'll come back to Bridgewater at the end to see what their result actually proves.
Open weights don’t mean free. They mean downloadable
The confusion is understandable, because one number in this space really is collapsing but that number describes one input to your bill, not your bill.
When Moonshot released Kimi K3 this month, it priced at $3 per million input tokens and $15 per million output. That’s roughly 40% below GPT-5.6 Sol and it’s 2-3x more expensive than Z.ai’s GLM-5.2, another open model. This is what’s causing the market to go frenzy.
You can, of course, run it yourself. That’s what “open” actually buys you. But K3 is a 2.8-trillion-parameter model, and self-hosting isn’t the only way to get it, so it’s worth separating two very different decisions before doing any math.
Open weights can reach you two ways: you self-host, or you buy them from a neocloud or API provider like Fireworks, Together, Groq, or Moonshot’s own API. Only the first one makes you own availability and accountability. The second is a lot closer to buying from OpenAI or Anthropic than the “open vs. closed” framing suggests: you’re still renting a bundle, just from a different, usually cheaper, vendor. The rest of this post walks through what that bundle contains and where self-hosting specifically, not “open weights” generally, changes the math.
For our billing agent: if you’re handling 50,000 disputes a month at maybe 3,000 tokens each including account history, policy documents, prior correspondence then you’re at 150 million tokens. Your 150M tokens could either put you comfortably above or meaningfully below the break even utilization of your GPU cluster. Which is itself the point: the “should I self-host” question isn't decidable from published numbers, only from your own.
What are you actually buying when you buy a token?
A token price looks like one number but underneath, it’s four things bundled together which behave completely differently depending on how you get your open weights.
1. Capability: is the model smart enough?
The question: can this model do the job at all?
For our billing agent: it needs to read a messy dispute, reconcile it against a payment ledger, apply a refund policy with exceptions in it, and produce a dollar figure. Drafting sympathetic prose is easy now and open models clear it comfortably. Getting the number right, every time, on the edge cases with partial refunds, prorated plan changes, a chargeback already in flight could be a gap that’s not too far from closing with frontier models.
It’s worth knowing why. A lot of the open-weight models that close the gap with a concept called distillation i.e. training a smaller model on the outputs of a larger one, so the student inherits the teacher’s behavior at a fraction of the training cost. The most useful description of what it does comes from Anthropic’s position on open weight models and why they want distillation policed noting that it’s far more compute-efficient than training from scratch.
2. Availability: will the GPUs be there at 9am?
The question: when your traffic spikes, does someone guarantee capacity?
For our billing agent: dispute volume isn’t flat, and it spikes on billing cycle dates, after a pricing change, and any time your payment processor has a bad afternoon. Those are precisely the moments when a slow or unavailable agent costs you the most. On an API, that variance is somebody else’s problem; you pay per token and capacity exists. Self-host and you own the gap between your peak and your average.
Here’s where the self-host/API-served split matters. Self-host, and you own the gap between your peak and your average. Buy K3 from Fireworks or Moonshot’s own API instead, and you’re back to renting availability, the same way you would from OpenAI: you pay per token, capacity exists, the variance is someone else’s problem. It’s not the same SLA a frontier lab offers, but it’s a real answer.
3. Compute: the silicon and the electricity
The question: what does it physically cost to run this request?
For our billing agent: A billing agent's "one request" isn't one token in, one token out. Reconciling a dispute case involves several tool calls and several context reloads per case. The token count per resolved dispute is a multiple of what a chat reply costs, even at the same per-token price.
That's why cheaper inference hasn't lowered anyone's bill. Prices per token fall, but multi-step agent workloads use more tokens per task than anyone budgeted for, so volume rises faster than price falls. And upstream, the hardware isn't getting cheaper to build in the first place. This one applies whether you self-host or buy from an API. Someone, somewhere, is still paying for the silicon per token you use.
4. Accountability: who’s on the hook when it breaks?
The question: at 2am, when the model starts hallucinating refund amounts, who fixes it?
For our billing agent: this is where the example stops being abstract. When the model approves a refund it shouldn’t have, or denies one it should have, somebody has to notice. On a frontier API you have an SLA, a status page, a compliance posture you can hand to your security reviewer, and a vendor whose job it is to catch degradation. Self-hosting pushes the evals, the on-call rotation, the model update decisions, the SOC 2 questionnaire, and the roughly one-third of a senior engineer permanently engaged.
One important scope note before you generalize any of this: the bundle applies the most for workloads that are synchronous, customer-facing, and compliance-sensitive which the billing agent is, deliberately, as an example. However, if your workload is offline or internal like nightly enrichment, batch classification, internal search reranking then the 2am-hallucination stakes and the peak-traffic availability problem mostly don’t apply, and accountability and availability stop being hard constraints.
So why are open-weight models getting popular and who actually wants them to win?
On July 24, Jensen Huang made the first post of his life on X. He used it to share a three-page letter called “Open Weights and American AI Leadership,” asking Washington not to restrict models you can download and run yourself.
Jensen has obvious reasons to want open weights unregulated and that doesn't make the letter wrong, but it means its recommendations are worth checking against what the people paying for compute are actually doing, not taking on the letter's authority alone. Let’s look at who’s actually giving away frontier-grade weights, and why.
Nvidia publishes Nemotron and convenes a coalition to publish more. Every open model downloaded is potential GPU demand, and Nemotron runs best on Nvidia silicon. There’s a defensive angle too: OpenAI, Anthropic, Google, Amazon and Meta are all building custom chips to escape Nvidia’s pricing, and those programs are funded out of model-layer margin.
Thinking Machines (Mira Murati’s lab) released its first foundation model, Inkling, under an Apache license this month, and said plainly it has no plans to monetize it. The revenue comes from Tinker, the fine-tuning API it shipped last October. The bet is that capability is a commodity and customization is scarce.
Reflection is selling accountability. Its pitch is American provenance, auditability, and a coding agent deployed into your own VPC. That’s a bet that regulated buyers will pay a premium for a frontier model that isn’t Chinese.
The Chinese labs like DeepSeek, Moonshot, Alibaba, Z.ai are commoditizing capability itself. Free weights erode the pricing power American labs need to fund everything else, buy global developer mindshare while trailing the frontier by three to six months, and route the world’s adoption through an increasingly domestic ecosystem.
Every player pushing open weights is also selling something the bundle framework says still matters. That’s not a reason to distrust the technology but it’s a reason to trust your own math over any single messenger’s framing of it.
So should you switch?
Here’s what I think happens next. The premium labs charge above raw compute compresses hard but not to zero, because reliability and safety posture are worth real money. The frontier labs hold the top of the market, where coding agents and hard reasoning will always pay for the best available model, and lose the boring middle. Which is why you’re seeing OpenAI and Anthropic push up into applications.
For almost every PM or Founder reading this, “switch to open models” is the wrong framing. The obvious one is “stop being unable to switch.” But how?
Run the bundle test on your own feature. For each of the four factors i.e capability, availability, compute, accountability ask whether open weights actually change your answer, and whether you'd be self-hosting or buying from an API-served open model. For a synchronous, compliance-sensitive workload like the billing agent, expect capability to move and the other three to hold firmer than the sticker price implies.
Route and don’t replace. This is what the teams actually saving money are doing, with numbers to show for it. Ramp built an internal router across OpenAI, Gemini, and open models and cut LLM costs about 30%. Cursor reported 30–50% savings routing away from its default frontier model with no drop in output quality.
For our billing agent: the split isn’t by ticket type, it’s by step. Classifying the dispute, pulling the relevant policy clause, summarizing the account history, and drafting the prose all go to an open model. Computing the refund amount and deciding whether policy permits it escalate to frontier. Most of your tokens are in the first group. Almost all of the risk is in the second.
Fix the swap cost before you chase the savings. If moving your primary model takes more than a day, that’s worth more this quarter than the 30% you might save by switching today. Put every call behind an OpenAI-compatible gateway. Build an eval harness against your own resolved disputes and not some leaderboards. Instrument token cost per feature the way you’d already instrument latency.
Only self-host when three things are true at once: you’re comfortably past the utilization crossover, your workload is stable enough to keep GPUs busy, and you have someone whose actual job is owning that. Two out of three isn’t enough. You likely would need all three and even then, buying the same open weights from an API provider is worth pricing out first, since it gets you most of the cost savings without taking on availability and accountability yourself.
Back to Bridgewater
So what actually happened with that 84.7% result? When you run it through the bundle, it stops looking like a capability story.
Bridgewater didn’t just download an open model and get a free win but instead they ran it through Tinker, Thinking Machines’ fine-tuning API. Bridgewater paid for the thing that made the model theirs i.e. they tuned on their own financial data, shaped to their own edge cases.
But here’s what the number actually tests: a fine-tuned open model against out-of-the-box frontier models but not necessarily against a fine-tuned frontier model. OpenAI and Anthropic both offer fine-tuning too, and nobody’s published that comparison. Until they do, the honest reading of Bridgewater’s result isn’t ‘open models are catching up,’ it’s ‘fine-tuning on your own data beats not fine-tuning’
That’s the frame worth carrying into your next planning cycle. Capability is getting cheap for everyone, including your competitors, which means it’s not where your product gets better. You already know that your evals, your data, and the workflow you’ve built around the model are the parts a model swap can’t carry away.