2a:null 2b:["$","title",null,{"children":"The Price of Performance: Evaluating LLM Cost vs. Accuracy | Scott King | Observable"}] 30:Tca7,

This scatter plot visualizes the relationship between the operational output cost of leading Large Language Models (LLMs) and their actual performance across key business metrics. By plotting the Overall Pass Rate (based on an average score threshold of >80) against the Output Cost per 1M tokens, the chart highlights the aggressive price scaling associated with proprietary "celebrity" models compared to high-value alternatives.

Key Insights

The Output Value Gap: While input costs are often comparable, output costs reveal the true operational expense of these models. Krista LLM demonstrates a significant advantage, maintaining an 83.4% pass rate at an industry-leading output cost of only $0.10 per 1M tokens.
The Efficiency Frontier: Open-source models like gpt-oss-120b (84.2% pass rate at $0.60) provide superior performance-to-price ratios compared to proprietary "mini" models with similar price points.
Premium Reasoning Costs: Top-tier performers such as gemini-3-pro-preview (87.5% pass rate) and claude-sonnet-4.5 (84.3% pass rate) sit at the far right of the scale, with output costs reaching $12.00 to $15.00 per 1M tokens.
Obsolescence in the Mid-Tier: Legacy models like gpt-3.5-turbo show a clear lack of utility in the modern landscape, appearing in the "low-value" zone with lower performance and higher output costs than several state-of-the-art alternatives.

Get the Full Report:
For a deeper dive into the methodology, model specifications, and how these rankings were determined, read the complete analysis here: Breaking the Celebrity LLM Monopoly

31:T689,data = [ { model: "gemini-3-pro-preview", passRate: 87.5, outputCost: 12.00, type: "Closed" }, { model: "o3-mini (Reasoning)", passRate: 87.0, outputCost: 4.40, type: "Closed" }, { model: "gpt-5.2", passRate: 85.7, outputCost: 14.00, type: "Closed" }, { model: "gpt-4.1-mini", passRate: 85.7, outputCost: 1.60, type: "Closed" }, { model: "claude-sonnet-4.5", passRate: 84.3, outputCost: 15.00, type: "Closed" }, { model: "gpt-oss-120b", passRate: 84.2, outputCost: 0.60, type: "Open" }, { model: "gpt-4.1", passRate: 83.8, outputCost: 8.00, type: "Closed" }, { model: "Krista LLM", passRate: 83.4, outputCost: 0.10, type: "Closed" }, { model: "Kimi-K2-Instruct", passRate: 83.3, outputCost: 2.50, type: "Open" }, { model: "claude-sonnet-4", passRate: 83.3, outputCost: 15.00, type: "Closed" }, { model: "gpt-oss-20b", passRate: 83.1, outputCost: 0.30, type: "Open" }, { model: "Llama-3.3-70B-Instruct", passRate: 81.7, outputCost: 0.90, type: "Open" }, { model: "gpt-5", passRate: 81.6, outputCost: 10.00, type: "Closed" }, { model: "gpt-5-mini", passRate: 81.5, outputCost: 2.00, type: "Closed" }, { model: "grok-4-1-fast", passRate: 81.2, outputCost: 0.50, type: "Closed" }, { model: "gpt-5.1", passRate: 81.1, outputCost: 10.00, type: "Closed" }, { model: "Grok-4-fast", passRate: 81.0, outputCost: 0.50, type: "Closed" }, { model: "Llama-4-Maverick", passRate: 80.6, outputCost: 0.88, type: "Open" }, { model: "gpt-4o", passRate: 80.2, outputCost: 10.00, type: "Closed" }, { model: "gpt-4o-mini", passRate: 74.6, outputCost: 0.60, type: "Closed" }, { model: "gpt-3.5-turbo", passRate: 64.2, outputCost: 1.50, type: "Closed" } ]32:[] 2e:{"id":"2d94335b9260baed","slug":"breaking_llm_monopoly_02_2026","title":"The Price of Performance: Evaluating LLM Cost vs. Accuracy","thumbnail":"424c8dd43b9b37aa7d6c9fa338bc348d9671a5995c57aa7abdf0472f30300b63","access_level":"public","create_time":"$D2026-02-25T19:36:00.539Z","update_time":"$D2026-02-25T20:17:39.603Z","view_time":null,"first_public_time":"$D2026-02-25T19:41:19.678Z","has_unpublished_edits":null,"fork":false,"stars":0,"starred":false,"owner":{"type":"team","id":"0b07b1112192b6dc","login":"thescottking","name":"Scott King","avatar_url":"https://avatars-next.observableusercontent.com/ff28697a5434b0dbce92e524330670d50030d1cbcf215484e350e257931e7060","tier":"starter_2024"},"version":39,"publish_version":null,"role":null,"authored":false,"authors":[{"type":"individual","id":"fbb7ff7aee948f4b","login":"thescottking","name":"Scott King","avatar_url":"https://avatars-next.observableusercontent.com/ff28697a5434b0dbce92e524330670d50030d1cbcf215484e350e257931e7060","tier":null}],"members":[{"type":"team","id":"0b07b1112192b6dc","login":"thescottking","name":"Scott King","avatar_url":"https://avatars-next.observableusercontent.com/ff28697a5434b0dbce92e524330670d50030d1cbcf215484e350e257931e7060","tier":"starter_2024","role":"editor"},{"type":"individual","id":"fbb7ff7aee948f4b","login":"thescottking","name":"Scott King","avatar_url":"https://avatars-next.observableusercontent.com/ff28697a5434b0dbce92e524330670d50030d1cbcf215484e350e257931e7060","tier":null,"role":"editor"}],"comments":[],"custom_thumbnail":false,"trashed":false,"lapsed":null,"subscribed":null,"forks":0,"collection":{"id":"6b360fd218bca288","slug":"krista-llm-research-2026","title":"Krista LLM Research 2026","owner":{"id":"0b07b1112192b6dc","login":"thescottking","name":"Scott King"}},"type":"notebook","body":{"titling":"auto","title":"The Price of Performance: Evaluating LLM Cost vs. Accuracy","cells":[{"id":25,"value":"# The Price of Performance: Evaluating LLM Cost vs. Accuracy","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":13,"value":"$30","pinned":false,"hidden":false,"mode":"html","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":36,"value":"# The Price of Performance: Evaluating LLM Cost vs. Accuracy","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":8,"value":"Plot.plot({\n grid: true,\n width: 900,\n height: 600,\n marginTop: 40,\n marginRight: 100,\n x: { \n label: \"Output Cost ($ per 1M tokens) →\", \n type: \"log\", \n domain: [0.08, 20]\n },\n y: { \n label: \"↑ Overall Pass Rate (%)\", \n domain: [60, 95]\n },\n color: {\n legend: true,\n domain: [\"Closed\", \"Open\"],\n range: [\"#2196F3\", \"#4CAF50\"]\n },\n marks: [\n Plot.ruleY([80], {strokeDasharray: \"4,4\", strokeOpacity: 0.5}), // Benchmark\n Plot.dot(data, {\n x: \"outputCost\",\n y: \"passRate\",\n fill: \"type\",\n r: 6,\n tip: true\n }),\n Plot.text(data, {\n x: \"outputCost\",\n y: \"passRate\",\n text: \"model\",\n dy: -14,\n fontSize: 10,\n fontWeight: \"bold\"\n })\n ]\n})","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":33,"value":"$31","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"}],"files":"$W32","stdlib":"1"}} 2f:"$2e:body"