JJoche Ojeda.NET · XAF · AI · architecture
Aug 5, 20269 min read/2026/08/05/open-weights-doesnt-mean-local/

Open Weights Doesn't Mean Local — and "Same Model" Doesn't Mean the Same Model

Open WeightsOpen SourceLocal LLMDeepSeekLlamaQuantization

Three words get used as if they were interchangeable, and they are not: open weights, open source, and local.

Untangling them started as vocabulary pedantry. It stopped being pedantry when I pulled the live provider data and found that "the same model" isn't even the same model.

They're independent properties

  • Open weights is a licensing and distribution property. The weights file is downloadable.
  • Local is a deployment location property. It runs on hardware you control.
  • Open source is a legal property covering code, training data, weights and license together. Almost nothing in this market qualifies, and it's the term people abuse most.

The first two are orthogonal, which means four quadrants — and all four are populated:

Remote Local
Open weights DeepSeek on DeepInfra, Together, Novita — the most common case by far llama.cpp, Ollama, LM Studio on your own box
Closed weights OpenAI, Anthropic — what everybody pictures Apple's on-device model, Gemini Nano on a Pixel

That top-right cell is the one people forget exists, and it's where most open-weight usage actually lives. When a team says "we moved to open models," nine times out of ten they mean they changed which company's API they call.

The bottom-right cell proves the axes are genuinely independent: a model running on the phone in your pocket, whose weights you will never see.

Open weights is not a privacy property

This is the expensive version of the mistake. Somebody picks an open-weight model believing they've solved data control, then calls it through a hosted API — and their data is going to a third party exactly as before. Just a different third party, often a younger company with a thinner compliance story than the incumbent they left.

The accurate statement is narrower and, I'd argue, more valuable:

Open weights doesn't give you privacy. It gives you the option of privacy.

The option is real. Almost nobody exercises it.

Then I looked at what "the same model" actually means

Here's where it stopped being a vocabulary post.

OpenRouter publishes, per model, every provider currently serving it and the exact terms each one serves it under. So I asked it about Llama 3.3 70B — one model, one name, one set of weights.

Thirteen providers. And they are not selling the same thing.

  • Four different quantization levels across those thirteen: bf16, fp16, fp8, and — my favourite — unknown.
  • A 7× spread in output price, from $0.32 to $2.25 per million tokens.
  • Five of the thirteen don't support structured outputs. DeepInfra, Novita, Cloudflare, Groq, Google.
  • Two won't accept a seed parameter. SambaNova and Together.

Same weights file. Wildly different product.

DeepSeek V3 0324 tells the same story in miniature, and more legibly because there are only four providers:

Provider Quantization Max output tokens In $/M Out $/M Structured outputs Seed
DeepInfra fp4 16,384 0.24 0.90 yes yes
SiliconFlow fp8 163,840 0.25 1.00 yes no
Novita fp8 65,536 0.27 1.12 no yes
Crusoe bf16 163,840 0.50 1.50 yes yes

Read the first and last rows together. The cheapest provider serves it at 4-bit. The most expensive serves it at bf16 — full precision. That price difference isn't a discount. You are buying different numerical precision under one product name, and the pricing is telling you so if you know where to look.

Now read the max-output column. DeepInfra caps completions at 16,384 tokens; SiliconFlow allows 163,840. That's a 10× difference on a field most people never check, and all four advertise the same 163,840 context length. A job that needs a long output doesn't degrade on the cheap provider — it fails.

(Data pulled 5 August 2026. It changes; see the bottom of this post for how to pull it yourself.)

What that breaks, concretely

Three of those columns aren't trivia. They break real code.

No seed means you cannot reproduce a run. My eval harness generates three variants per task and requires all three to pass, precisely so a solver that hardcodes a cell coordinate gets caught. On a provider that ignores seeds, that discipline is unavailable. I can't even run my own methodology there.

No structured outputs means your JSON mode silently isn't. Five of thirteen Llama providers. If your code path assumes a schema-constrained response, moving to one of those isn't a config change, it's a rewrite.

Undisclosed quantization means you don't know what you bought. That unknown label is the honest one — at least it admits it. The concerning case is a provider that publishes nothing at all and serves you 4-bit while a competitor serves 16-bit at a similar price.

And quantization is not cosmetic. It's a lossy compression of the model, and the loss shows up first on exactly the things that are hardest to spot in a demo: long-context reasoning, arithmetic, rare tokens, and the tail of the distribution. Which is to say — it shows up in production, not in your first ten test prompts.

The model name is not a grade

Regular readers will see where this lands.

Last post I argued that tokens are commoditizing, and that the mechanism is the grading standard: WTI is tradeable because someone wrote a specification — a thousand barrels, this gravity, this sulfur content, delivered at Cushing. The document is what creates the fungibility.

I thought open weights would be the strongest counterexample to my own caution. If the weights are downloadable, many suppliers can serve the identical artifact — you don't need an eval to know two sellers are offering the same thing, you can hash the file. It should be the most fungible good in the entire market.

It isn't, and the reason is exactly the one from the commodity post. "Llama 3.3 70B" is a name, not a specification. It tells you which weights went in. It says nothing about the precision they're served at, how much output you're allowed, which parameters are honoured, or whether you can reproduce a result.

That's not a grade. That's the word "oil."

So the conclusion survives contact with the case that should have broken it: even when the underlying good is provably identical, you still need the grading standard, because what's sold to you is the serving, not the weights. Downloadable weights raise the ceiling on fungibility. They don't deliver it.

One more wrinkle worth knowing: open weights doesn't even guarantee you have multiple suppliers. Qwen3 235B, when I checked, had exactly one provider. Openly licensed, and a single seller — which is a monopoly with a permissive license attached, not a competitive market.

Local is a capex decision, not a savings plan

The other half of the confusion is the belief that local means cheap.

A GPU that comfortably runs a 70B model costs thousands, and an idle GPU is pure loss. Serving providers run utilization you will never match, which is the commodity cost-curve argument from the last post pointed back at you: the marginal producer with scale beats you, and you are not the marginal producer.

Local genuinely wins on several things, and they're worth real money:

  • Data that contractually cannot leave your infrastructure
  • Predictable spend instead of per-call variance
  • No rate limits, which — as I've argued before — is the scarcity that actually bites
  • Offline operation
  • A model version that nobody can retire out from under you

Notice that "it's cheaper" isn't on that list at ordinary volumes, even though it's the reason people cite most often. At high, steady utilization the math flips. At bursty or low volume it doesn't, and buying a GPU to save money on inference is usually a decision made before the arithmetic.

The business case nobody states correctly

If you want one reason to care about open weights, it isn't privacy and it isn't cost. It's this:

Open weights is insurance against deprecation.

Closed API models get retired on the vendor's schedule. When that happens, the model you qualified is gone, every eval result you have evaporates, and you re-certify a replacement on someone else's timeline — usually with a deadline attached. With open weights, the artifact you certified stays available. You can pin it, host it, and keep running it past the vendor's end-of-life.

That has nothing to do with running locally, and nothing to do with saving money. It's continuity risk, and it's the only argument in this whole area that survives contact with a CFO.

But it comes with a condition, and this is the part I'd underline. An option you can't execute is worth nothing. If you've never actually migrated, and you don't have a harness that can validate a swap on the day you need it, then "we use open models" is a feeling, not a hedge — and you're paying real complexity for insurance you've never tested.

The eval is what converts open weights from a comfort into an executable option. Same instrument as last time, second job.

Verify this yourself

All the provider data above comes from one public endpoint, no key required:

curl -s "https://openrouter.ai/api/v1/models/meta-llama/llama-3.3-70b-instruct/endpoints" | jq '.data.endpoints[] | {provider: .provider_name, quantization, max_completion_tokens, out_price: .pricing.completion, structured: (.supported_parameters | index("structured_outputs") != null)}'

Swap the model ID for whatever you're actually running. I'd suggest doing it for the model in your production config right now, before you finish reading — the quantization and max_completion_tokens fields in particular tend to surprise people who assumed a name was a contract.

So

Open weights, open source and local are three properties, not one. Most open-weight usage is remote. Open weights buys you the option of privacy, not privacy. And "the same model" from thirteen vendors is thirteen products that share a filename.

None of that is an argument against open weights. I use them, they're the reason the cheap end of this market got cheap, and the deprecation insurance is genuinely worth having.

It's an argument against believing the name told you what you bought. Which is, once again, the same finding: build the ruler first. The label on the sack has never been the specification, in this industry or any older one.