Agents Lookup

Learn / LLMs

Open-weight vs hosted models

When to rent a model through an API and when to run open weights yourself, with the licence, hardware and privacy trade-offs spelled out.

Updated 2026-11-19. "(source N)" refers to the numbered list at the end.

Two ways to get a model

A hosted model lives on the vendor's servers. You send text over an API and pay per token. Claude, GPT and Gemini work this way.

An open-weight model publishes its trained weights, the large files that hold what the model learned. You can download them and run the model on your own machine or on a server you rent. Llama, Qwen, DeepSeek, Gemma and some Mistral models work this way. OpenAI also publishes two open models, gpt-oss-120b and gpt-oss-20b (source 5).

Open weights are not the same as open source. You usually get the finished model, not the training data or the full recipe, and the licence sets the rules.

Licences differ more than you think

Read the licence before you build a product on a model. A few current examples:

The hardware reality

Open does not mean small. Many top open models are huge, and running them well takes data-centre graphics cards.

DeepSeek V4.1 Flash has hundreds of billions of parameters, though only about 8 to 16 billion are active for each token because of how it is built (source 3). Llama 4 Scout has 109 billion parameters in total. Meta says it fits on a single H100 card, but only after squeezing the weights down to 4-bit precision (source 1). An H100 is a server card, not something in a laptop.

Smaller open models are a different story. Tools like Ollama let you download and run a model on an ordinary computer from the command line (source 10). Its library shows the most pulled models include Llama 3.1, Qwen and Gemma, and the newest Qwen3.8 builds are already listed (source 6). For serving a model to many users on your own servers, the open-source vLLM engine is a common choice (source 7).

Why pick hosted

For most small teams, a hosted API is the sensible default.

Why pick open weights

There are good reasons to run your own.

The middle path

You do not have to choose all or nothing. Many hosts rent out open models by the token, so you get open-model prices without running servers. DeepSeek's own API, for example, serves V4.1 Flash at $0.15 to $0.30 per million input tokens and $0.60 to $1.20 per million output, with half-price off-peak hours (source 9). Mistral's platform serves its own open models and third-party ones like GLM and DeepSeek (source 4).

Check where such a host is based and what its data terms say. A cheap API in another country may not meet your privacy needs, even if the model itself is open.

Be careful with free tiers in general. Google's terms, for instance, say prompts sent on the free Gemini API tier may be used to improve its products and read by human reviewers (source 8). Paid tiers are usually stricter.

A quick decision guide

Whichever you pick, keep your app able to switch. Serving engines such as vLLM and SGLang can expose an OpenAI-compatible API (source 3), so moving between hosted and self-run models can be a configuration change rather than a rewrite.

Sources

Written by Agents Lookup in our own words. Facts come from these pages: