Cost vs. pass rate frontier

Each point is one model/backend arm. The Y-axis is overall pass rate. The X-axis is cost per benchmark attempt.

Cost uses adjusted known cost: recorded cost plus reconstructed missing-cost rows where usage/pricing evidence exists. This avoids undercounting failed or errored attempts that consumed tokens.

Other arms Pareto frontier
$0.5 $1.0 $1.5 $2.0 $2.5 $3.0 20% 30% 40% 50% 60% 70% Cost per benchmark attempt (USD) Pass rate router-gpt-5.5 router-deepseek-flash router-anthropic-fable-5 router-anthropic-opus router-gpt-5.4 router-glm-5.2 router-gemini-3.1-pro router-deepseek-pro router-glm-5.1 router-grok-build-0.1 router-qwen-3.7-plus router-kimi-k2.6 router-anthropic-sonnet router-anthropic-haiku-sanitized router-gemini-flash

Hover over a point for model, provider, pass rate, adjusted total cost, cost per clean success, unclean spend, and cost-confidence details.