Agents Lookup

Learn / LLMs

Choosing a model: cost, speed, context window, privacy

Four questions that decide which model fits your agent: what it costs per task, how fast it must be, how much it must read, and where your data goes.

Updated 2026-11-19. "(source N)" refers to the numbered list at the end.

Start with the task, not the leaderboard

Benchmarks tell you which model is smartest on average. They do not tell you which one is best for your refund emails or your product feed. OpenAI's advice to builders is a good rule: build your first version with the most capable model to set a quality bar, then swap in smaller models and see where they still pass (source 5).

To do that you need a small test set: 20 to 50 real examples with known good answers. Anthropic suggests starting with cases based on things that actually went wrong (source 9). Then judge each model on four things.

1. Cost per task, not per token

Prices are quoted per million tokens, but you pay per task. A support agent that loops eight times and re-reads a long tool list each time can use 100,000 tokens or more for one ticket.

The spread between tiers is wide. On Anthropic's list, Claude Opus 5.5 costs $4 per million input tokens and Haiku 5.5 starts at $0.10 (source 1). OpenAI's GPT-6.2 Astra is $8 in and GPT-6 Luna is $0.10 (source 3).

Read the fine print too.

2. Speed

For a live chat, a reply that takes 30 seconds feels broken. For an overnight report, nobody cares. Decide which you are building.

Vendors give rough guides. Anthropic labels Haiku 5.5 as its fastest model and Sonnet 5.5 as fast, with Opus and Fable as moderate (source 1). Smaller models are almost always quicker.

You can also pay for speed or save by waiting. OpenAI sells a Fast tier at double the standard rate, and both OpenAI and Anthropic offer batch processing at half price if you can wait up to 24 hours (source 3, source 2). Anthropic's fast mode gives up to 2.5 times quicker output on some models at a premium price (source 2).

3. Context window

The context window is how much text the model can hold at once: your instructions, tool definitions, documents and the conversation so far. All current Claude models offer 1 million tokens, which Anthropic says is about 555,000 words with its current tokenizer (source 1). OpenAI's flagships offer about 1.05 million (source 3), and Gemini 4 models 1 million (source 4).

Big windows are useful but not free. Every token you send is billed, every time. A large window is a ceiling, not a target. Send only what the task needs, and use caching for the parts that repeat.

4. Privacy and where your data goes

This is the question small firms skip and later regret. Check three things: does the vendor train on your data, how long does it keep it, and who can see it.

The paid business APIs from the big three do not train on your prompts by default.

Sensitive data

If you handle health, legal or card data, you may need more: a signed data processing agreement, a specific region, or a model you host yourself. Our article on open-weight models covers the self-hosted route.

A simple scorecard

Put your shortlist in a table with four columns: pass rate on your test set, cost per task, typical response time, and data terms. Drop any model that fails the privacy check first. Then pick the cheapest one that clears your quality bar.

Re-run the scorecard every few months. New models arrive often and prices move, so the right choice in November may not be the right choice in March.

Sources

Written by Agents Lookup in our own words. Facts come from these pages: