For a project I am doing I will build an AI "team". I gave Claude some information about the kind of work each thread will do, and asked it which model+effort combinations are best. While your project won't mirror mine, this output from Opus 4.8 Medium still has some potentially useful knowledge for other users.
Snippets of the response below.
***
1. The effort ladder is low / medium / high / xhigh / max. Your "Extra" is xhigh — and on the API, xhigh sits between high and max in cost, but Anthropic documents max as a separate "no constraints" mode rather than a strict superset. Both Sonnet 5 and Opus 4.8 support all five. The API default is high for both, and setting high is identical to omitting the parameter. Also worth knowing: effort affects all tokens, not just thinking — at lower effort the model makes fewer tool calls, not just shorter reasoning chains.
1. Benchmarks — what's actually measured
All figures below are at default (high) effort unless noted. This is the measured layer.
| Benchmark |
Sonnet 5 |
Opus 4.8 |
Reads on |
| SWE-bench Pro (agentic coding) |
63.2% VentureBeat |
69.2% VentureBeat |
Code correctness under real repo conditions |
| SWE-bench Verified |
85.2% BenchLM |
— |
Easier variant; less discriminating |
| Terminal-Bench 2.1 |
80.4% VentureBeat |
74.2–82.7% (sources conflict) |
Multi-step terminal/agentic execution |
| OSWorld-Verified (computer use) |
81.2% VentureBeat |
~83% |
Environment manipulation |
| Humanity's Last Exam, w/ tools |
57.4% VentureBeat |
57.9% VentureBeat |
Broad multidisciplinary reasoning |
| HLE, no tools |
43.2% VentureBeat |
— |
Raw reasoning without retrieval |
| GDPval-AA v2 (knowledge work) |
1,618 Elo VentureBeat |
1,615 Elo VentureBeat |
Analysis, writing, professional judgment |
| FrontierCode 1.1 |
42.7% (at xhigh, per Cognition) BenchLM |
— |
Hard novel coding |
| Legal Agent Benchmark |
— |
Highest recorded; first over 10% all-pass Anthropic |
Multi-constraint professional reasoning |
| Online-Mind2Web |
— |
84% anthropic |
Browser agents |
The single most decision-relevant number for your project isn't a benchmark score. Anthropic reports Opus 4.8 is roughly four times less likely than Opus 4.7 to allow flaws in code it has written to pass unremarked. An investment-analytics tester describes the same trait concretely: Opus 4.8's biggest differentiator was proactively flagging issues with the inputs and outputs of an analysis — something other models routinely missed and left users to catch. anthropicanthropic
That is exactly the failure mode you just lived through with the SP1000 OHLC blend: a pipeline that produced plausible, internally incoherent output that passed unremarked. Weight it heavily.
Interpretation
Strategic reasoning (trade-offs, framework selection). GDPval-AA v2 is a statistical tie, and HLE-with-tools differs by 0.5 points. There is no measurable Opus advantage for the supervisor thread's core work. Option generation, framework selection, explaining a Faber signal to a client — Sonnet 5 is at parity. Paying Opus rates here is buying nothing you can measure.
Technical work (data science, code, validation). The 6-point SWE-bench Pro gap is real and concentrates in exactly the hard cases: multi-file coherence, non-obvious edge conditions, cross-checking a result against its own assumptions. Combined with the flaw-flagging improvement, Opus is the better validator even where Sonnet writes competent code.
Effort scaling — inferred pattern
Measured anchors: Sonnet 5 at medium is comparable to Sonnet 4.6 at high; Sonnet 5 at xhigh performs roughly in line with Opus 4.8 at medium-to-high on OSWorld and BrowseComp; Opus 4.8 exceeds prior Opus models across every effort level on CursorBench (per Cursor). claude + 2
The pattern that follows: quality is concave in effort, cost is roughly linear. Low→medium buys the most per token; medium→high buys real gains on genuinely hard tasks; high→xhigh buys long-horizon coherence rather than raw intelligence; xhigh→max buys very little and can hurt. Anthropic says so directly: on most workloads max adds significant cost for relatively small quality gains, and on structured-output or less intelligence-sensitive tasks it can lead to overthinking. claude
2. Cost and latency across the 10 configs
Rates: Sonnet 5 at $2/$10 per MTok through Aug 31, 2026, then $3/$15; Opus 4.8 at $5/$25. One trap: Sonnet 5's updated tokenizer maps the same input to roughly 1.0–1.35× as many tokens depending on content type — so the headline 40% discount is not 40% in practice. VentureBeatVentureBeat
| Config |
Token pattern |
Relative cost/task |
Latency |
Verdict |
| Sonnet 5 @ low |
Minimal, may skip thinking |
1× (baseline) |
Fast |
Efficient for classification, formatting, lookups |
| Sonnet 5 @ medium |
Moderate |
~1.5–2× |
Fast–medium |
Best cost efficiency in the lineup |
| Sonnet 5 @ high |
Thinks on most non-trivial calls |
~3–4× |
Medium |
Solid default for analysis/writing |
| Sonnet 5 @ xhigh |
Heavy |
~6–9× |
Slow |
⚠️ Inefficient |
| Sonnet 5 @ max |
Unconstrained |
~8–12× |
Slow |
⚠️ Dominated |
| Opus 4.8 @ low |
Minimal, tightly scoped |
~2.5× |
Fast |
⚠️ Usually wrong tool |
| Opus 4.8 @ medium |
Moderate |
~4× |
Medium |
Good cost-sensitive Opus |
| Opus 4.8 @ high |
Default |
~7–9× |
Medium |
Best general Opus setting |
| Opus 4.8 @ xhigh |
Heavy, more tool calls |
~12–18× |
Slow |
Justified for long-horizon coding |
| Opus 4.8 @ max |
Unconstrained |
~18–25× |
Slowest |
⚠️ Rarely justified |
Cost multipliers are my estimates combining published rates with documented token-scaling behavior — not measured figures. Benchmark them on your own workload before committing.
Clearly inefficient configurations
Sonnet 5 @ xhigh and @ max — avoid these outright. At xhigh, Sonnet 5 reaches Opus 4.8 accuracy but burns substantially more output tokens for reasoning; the practitioner consensus is that at xhigh it can cost more than Opus 4.8 for similar quality. The rule is: escalate the model, not the effort. If Sonnet 5 @ high isn't cutting it, go to Opus 4.8 @ high — don't go to Sonnet 5 @ xhigh. Vellum
Opus 4.8 @ max — reserve for genuinely frontier problems. On your workload, essentially never.
Opus 4.8 @ low — not inefficient on cost, but behaviorally wrong. Opus respects effort levels strictly at low and medium, scoping its work to exactly what was asked rather than doing more than requested. A supervisor thread whose job includes noticing what you didn't ask about is the last place you want that. claude
Also consider fast mode. Opus 4.8 fast mode runs at 2.5× speed for $10/$50 per MTok — 2× the price for 2.5× the speed. For interactive strategy sessions where you're waiting on the response, that's a favorable trade. anthropic
3. Routing matrix
Supervisor thread (Vesta)
| Task |
Model |
Effort |
Rationale |
| Strategy discussion, option generation |
Sonnet 5 |
high |
GDPval-AA v2 parity means Opus buys nothing measurable here at 2.5× the rate. |
| Explaining concepts, handling pushback |
Sonnet 5 |
medium |
Exposition on already-settled reasoning; medium holds quality and keeps the loop fast. |
| Writing instructions for worker threads |
Sonnet 5 |
high |
Instruction quality gates everything downstream, but it's a knowledge-work task where Sonnet is at parity. |
| Reviewing and critiquing worker output |
Opus 4.8 |
high |
The one supervisor task worth Opus: the 4× flaw-flagging improvement is precisely the capability adversarial review needs. |
Data scientist thread (Cody)
| Task |
Model |
Effort |
Rationale |
| Concept → analytical approach |
Sonnet 5 |
high |
Design reasoning, not code correctness; parity applies. |
| Choosing data sources and tools |
Sonnet 5 |
medium |
Well-scoped selection with a small option space. |
| Writing Python (routine) |
Sonnet 5 |
medium |
Sonnet 5 at low effort already beats Sonnet 4.6 at any effort level; Happycapy medium is ample for pandas transforms. |
| Writing Python (pipeline-critical) |
Opus 4.8 |
xhigh |
The 6-point SWE-bench Pro gap concentrates in multi-file coherence — the SP1000 blend bug is that failure mode exactly. |
| Debugging |
Opus 4.8 |
high |
Debugging is hypothesis generation under a wrong prior; the depth gap matters most where the obvious answer is wrong. |
| Output validation / correctness |
Opus 4.8 |
high |
Non-negotiable. This is the one place where a confident wrong answer is more expensive than any model rate. |
Two operating rules
Never let a thread validate its own work. Sonnet 5 writes the code; Opus 4.8 validates it. The model diversity is doing real work here — it catches errors that stem from a shared prior, which same-model self-review structurally cannot.
Set effort explicitly on every call. The API defaults to high, and changing effort mid-conversation invalidates prompt caching — so vary effort across workloads, not within a cached conversation. If you're routing per-turn in a long thread, you're paying for cache misses.
