Choosing a model: cost, speed, context window, privacy
Four questions that decide which model fits your agent: what it costs per task, how fast it must be, how much it must read, and where your data goes.
Updated 2026-11-19. "(source N)" refers to the numbered list at the end.
Start with the task, not the leaderboard
Benchmarks tell you which model is smartest on average. They do not tell you which one is best for your refund emails or your product feed. OpenAI's advice to builders is a good rule: build your first version with the most capable model to set a quality bar, then swap in smaller models and see where they still pass (source 5).
To do that you need a small test set: 20 to 50 real examples with known good answers. Anthropic suggests starting with cases based on things that actually went wrong (source 9). Then judge each model on four things.
1. Cost per task, not per token
Prices are quoted per million tokens, but you pay per task. A support agent that loops eight times and re-reads a long tool list each time can use 100,000 tokens or more for one ticket.
The spread between tiers is wide. On Anthropic's list, Claude Opus 5.5 costs $4 per million input tokens and Haiku 5.5 starts at $0.10 (source 1). OpenAI's GPT-6.2 Astra is $8 in and GPT-6 Luna is $0.10 (source 3).
Read the fine print too.
- Haiku 5.5 charges more once a prompt goes over 100,000 tokens: $0.50 in instead of $0.10 (source 2).
- OpenAI's headline prices cover prompts up to 272,000 input tokens (source 3).
- Google's Gemini 4 Carbon and Argon double in price on 1 December 2026, and Gemini 3.8 Flash doubles on 1 January 2027 (source 4).
- Asking Anthropic to keep processing in the US adds 10% to the bill on newer models (source 2).
2. Speed
For a live chat, a reply that takes 30 seconds feels broken. For an overnight report, nobody cares. Decide which you are building.
Vendors give rough guides. Anthropic labels Haiku 5.5 as its fastest model and Sonnet 5.5 as fast, with Opus and Fable as moderate (source 1). Smaller models are almost always quicker.
You can also pay for speed or save by waiting. OpenAI sells a Fast tier at double the standard rate, and both OpenAI and Anthropic offer batch processing at half price if you can wait up to 24 hours (source 3, source 2). Anthropic's fast mode gives up to 2.5 times quicker output on some models at a premium price (source 2).
3. Context window
The context window is how much text the model can hold at once: your instructions, tool definitions, documents and the conversation so far. All current Claude models offer 1 million tokens, which Anthropic says is about 555,000 words with its current tokenizer (source 1). OpenAI's flagships offer about 1.05 million (source 3), and Gemini 4 models 1 million (source 4).
Big windows are useful but not free. Every token you send is billed, every time. A large window is a ceiling, not a target. Send only what the task needs, and use caching for the parts that repeat.
4. Privacy and where your data goes
This is the question small firms skip and later regret. Check three things: does the vendor train on your data, how long does it keep it, and who can see it.
The paid business APIs from the big three do not train on your prompts by default.
- Anthropic says it does not use inputs or outputs from its commercial products, including the API, to train models unless you give feedback or opt in (source 6).
- OpenAI says API data is not used for training unless you opt in. It keeps API data for up to 30 days for abuse checks, and zero data retention needs approval (source 7).
- Google is the one to watch. On the free tier of the Gemini API, Google may use your prompts to improve its products and human reviewers may read them. On the paid tier it does not (source 8).
Sensitive data
If you handle health, legal or card data, you may need more: a signed data processing agreement, a specific region, or a model you host yourself. Our article on open-weight models covers the self-hosted route.
A simple scorecard
Put your shortlist in a table with four columns: pass rate on your test set, cost per task, typical response time, and data terms. Drop any model that fails the privacy check first. Then pick the cheapest one that clears your quality bar.
Re-run the scorecard every few months. New models arrive often and prices move, so the right choice in November may not be the right choice in March.
Sources
Written by Agents Lookup in our own words. Facts come from these pages:
- 1. Anthropic: Models overview: platform.claude.com/docs/en/about-claude/models/overview
- 2. Anthropic: Pricing: platform.claude.com/docs/en/about-claude/pricing
- 3. OpenAI: API pricing: developers.openai.com/api/docs/pricing
- 4. Google AI Studio: Gemini API pricing: aistudio.google.com/docs/pricing
- 5. OpenAI: A practical guide to building agents: cdn.openai.com/business-guides-and-resources/a-practical-guide-to-building-agents.pdf
- 6. Anthropic Privacy Center: Is my data used for model training?: privacy.claude.com/en/articles/7996868-is-my-data-used-for-model-training
- 7. OpenAI: Your data: developers.openai.com/api/docs/guides/your-data
- 8. Google AI Studio: Gemini API additional terms of service: aistudio.google.com/docs/terms
- 9. Anthropic: Demystifying evals for AI agents: www.anthropic.com/engineering/demystifying-evals-for-ai-agents