{
 "_comment": "Two model tiers. standard: the models a buyer meets on a free plan, fourteen from twelve labs (OpenAI and Meta have two each since the October 2026 Edition: GPT-6 Luna beside GPT-5.4 mini, Muse Glimmer 30B beside Llama 4 Maverick); the public index runs on them from the October 2026 reset. expanded: the flagship set the September 2026 Edition ran on, and Muse Spark 1.3 from the October 2026 Edition, seven models; it runs per category for subscribers. Version strings were verified against each lab live model list on 2026-09-08 (expanded) and 2026-09-14 (free, direct models); gateway model ids and hosts were confirmed by a smoke through the gateway on 2026-09-14. A change to any string is a new Models row and triggers noise floor recalibration for that model. transport direct calls the lab; transport gateway calls Vercel AI Gateway with the provider pinned and the gateway search tool, because those labs offer no search tool of their own.",
 "judge": {
  "model": "claude-opus-5",
  "note": "It runs with reasoning effort set to low and a fixed output schema, so the same answer text produces the same labels every time.",
  "input_price_per_m": 5,
  "output_price_per_m": 25,
  "cost_per_answer_measured": 0.027,
  "cost_note": "Measured over the September 2026 Edition: about 2.7 cents an answer at low effort, 3,200 input and 500 output tokens on average, whatever model wrote the answer. Re-judged on 2026-09-14 against Sonnet 5 (72% first-choice set agreement) and Haiku 4.5 (65%) with Opus 5 reproducing itself at 93%: the judge stays Opus 5."
 },
 "judge_default": "student",
 "models": [
  {
   "model_key": "anthropic-current",
   "lab": "anthropic",
   "model": "claude-opus-5",
   "generation": "current",
   "endpoint": "https://api.anthropic.com/v1/messages",
   "search": "web_search_20260209",
   "input_price_per_m": 5,
   "output_price_per_m": 25,
   "search_price_per_call": 0.01,
   "tier": "expanded",
   "transport": "direct"
  },
  {
   "model_key": "anthropic-prior",
   "lab": "anthropic",
   "model": "claude-opus-4-8",
   "generation": "prior",
   "endpoint": "https://api.anthropic.com/v1/messages",
   "search": "web_search_20260209",
   "input_price_per_m": 5,
   "output_price_per_m": 25,
   "search_price_per_call": 0.01,
   "tier": "expanded",
   "transport": "direct"
  },
  {
   "model_key": "openai-current",
   "lab": "openai",
   "model": "gpt-6-astra",
   "generation": "current",
   "endpoint": "https://api.openai.com/v1/responses",
   "search": "web_search",
   "input_price_per_m": 10,
   "output_price_per_m": 50,
   "search_price_per_call": 0.01,
   "tier": "expanded",
   "transport": "direct"
  },
  {
   "model_key": "openai-prior",
   "lab": "openai",
   "model": "gpt-5.6-sol",
   "generation": "prior",
   "endpoint": "https://api.openai.com/v1/responses",
   "search": "web_search",
   "input_price_per_m": 4,
   "output_price_per_m": 20,
   "search_price_per_call": 0.01,
   "tier": "expanded",
   "transport": "direct"
  },
  {
   "model_key": "google-single",
   "lab": "google",
   "model": "gemini-3.1-pro-preview",
   "generation": "single",
   "endpoint": "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.1-pro-preview:generateContent",
   "search": "google_search",
   "input_price_per_m": 2,
   "output_price_per_m": 12,
   "search_price_per_call": 0.014,
   "note": "Switched from gemini-3.8-flash on 2026-09-08: Flash never invoked Google Search in testing, Pro grounds. Preview build.",
   "tier": "expanded",
   "transport": "direct"
  },
  {
   "model_key": "challenger-single",
   "lab": "challenger",
   "model": "sonar-pro",
   "generation": "single",
   "endpoint": "https://api.perplexity.ai/chat/completions",
   "search": "native",
   "input_price_per_m": 3,
   "output_price_per_m": 15,
   "search_price_per_call": 0.01,
   "tier": "expanded",
   "transport": "direct"
  },
  {
   "model_key": "meta-muse-spark",
   "lab": "meta",
   "provider": "meta",
   "model": "meta/muse-spark-1.3",
   "generation": "current",
   "tier": "expanded",
   "transport": "gateway",
   "endpoint": "https://ai-gateway.vercel.sh/v1/chat/completions",
   "search": "vercel:perplexity_search",
   "input_price_per_m": 1.25,
   "output_price_per_m": 4.25,
   "search_price_per_call": 0.005,
   "note": "Muse Spark 1.3, Meta's flagship from Meta Superintelligence Labs, closed weights, released 2026-09-02 at $1.25 in, $4.25 out per million, served by Meta on the gateway. Tool use, so the client search loop applies. Smoke 2026-09-30 on customer data platforms, mid-market, six prompts: searched on 6 of 6, 3.5 searches an answer, about $0.038 a pair, 16.5 sources cited an answer, 100 s; version string meta/muse-spark-1.3 via meta. Joined the expanded tier for the October 2026 Edition, decided 2026-09-30 on the smoke.",
   "smoke_cost_per_pair": 0.038,
   "since": "2026-10"
  },
  {
   "model_key": "anthropic-small",
   "lab": "anthropic",
   "model": "claude-haiku-4-5",
   "generation": "small",
   "tier": "standard",
   "transport": "direct",
   "endpoint": "https://api.anthropic.com/v1/messages",
   "search": "web_search_20250305",
   "input_price_per_m": 1,
   "output_price_per_m": 5,
   "search_price_per_call": 0.01,
   "note": "Haiku 4.5 takes the 20250305 web search tool; the 20260209 tool needs programmatic tool calling it does not support.",
   "smoke_cost_per_pair": 0.024
  },
  {
   "model_key": "openai-small",
   "lab": "openai",
   "model": "gpt-5.4-mini",
   "generation": "small",
   "tier": "standard",
   "transport": "direct",
   "endpoint": "https://api.openai.com/v1/responses",
   "search": "web_search",
   "input_price_per_m": 0.25,
   "output_price_per_m": 2,
   "search_price_per_call": 0.01,
   "smoke_cost_per_pair": 0.013
  },
  {
   "model_key": "google-small",
   "lab": "google",
   "model": "gemini-3.5-flash",
   "generation": "small",
   "tier": "standard",
   "transport": "direct",
   "endpoint": "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.5-flash:generateContent",
   "search": "google_search",
   "input_price_per_m": 0.3,
   "output_price_per_m": 2.5,
   "search_price_per_call": 0.014,
   "note": "Chosen 2026-09-14: 3.6, 3.7 and 3.8 Flash answered without invoking Google Search in testing; 3.5 Flash and 3.1 Flash-Lite ground on every prompt tried.",
   "smoke_cost_per_pair": 0.023
  },
  {
   "model_key": "challenger-small",
   "lab": "challenger",
   "model": "sonar",
   "generation": "small",
   "tier": "standard",
   "transport": "direct",
   "endpoint": "https://api.perplexity.ai/chat/completions",
   "search": "native",
   "input_price_per_m": 1,
   "output_price_per_m": 1,
   "search_price_per_call": 0.005,
   "smoke_cost_per_pair": 0.005
  },
  {
   "model_key": "xai-small",
   "lab": "xai",
   "provider": [
    "xai",
    "vertex"
   ],
   "model": "spacexai/grok-4.20-non-reasoning",
   "generation": "small",
   "tier": "standard",
   "transport": "gateway",
   "endpoint": "https://ai-gateway.vercel.sh/v1/chat/completions",
   "search": "vercel:perplexity_search",
   "input_price_per_m": 1.25,
   "output_price_per_m": 2.5,
   "search_price_per_call": 0.005,
   "note": "Grok 4.20 Non-Reasoning from the November 2026 Edition, in place of Grok 4.1 Fast (spacexai/grok-4.1-fast-non-reasoning), which the gateway stopped serving after 5 October 2026 (404 model_not_found on every call of the November runs). xAI's API has no fast or mini text model any more; Grok 4.20 and 4.3 are its lowest-priced general models at $1.25 in and $2.50 out per million below 200k tokens, and the non-reasoning variant is the nearest to 4.1 Fast. Served by xAI on the gateway, Vertex AI second. Grok 4.1 Fast answered every edition to October 2026 and its rows keep its version string. Smoke 2026-11-01 (mai-20261101-0920-smoke, c01 mid-market): six of six searched, 5 to 23 sources an answer, a first choice on every ask but the avoid ask, no thinking tokens, the distilled judge read all six, about $0.018 a pair.",
   "since": "2026-11",
   "smoke_cost_per_pair": 0.018
  },
  {
   "model_key": "mistral-small",
   "lab": "mistral",
   "provider": "mistral",
   "model": "mistral/mistral-small",
   "generation": "small",
   "tier": "standard",
   "transport": "gateway",
   "endpoint": "https://ai-gateway.vercel.sh/v1/chat/completions",
   "search": "vercel:perplexity_search",
   "input_price_per_m": 0.1,
   "output_price_per_m": 0.3,
   "search_price_per_call": 0.005,
   "note": "Smoke 2026-09-14: searched, served by mistral, about $0.016 a pair (three searches).",
   "smoke_cost_per_pair": 0.016
  },
  {
   "model_key": "deepseek-small",
   "lab": "deepseek",
   "provider": [
    "fireworks",
    "deepinfra",
    "baseten"
   ],
   "model": "deepseek/deepseek-v4-flash",
   "generation": "small",
   "tier": "standard",
   "transport": "gateway",
   "endpoint": "https://ai-gateway.vercel.sh/v1/chat/completions",
   "search": "vercel:perplexity_search",
   "input_price_per_m": 0.06,
   "output_price_per_m": 0.18,
   "search_price_per_call": 0.005,
   "note": "DeepSeek does not serve its own model through the gateway. Fireworks is the first host; DeepInfra and Baseten follow only when Fireworks reports no capacity, and the host that answered is recorded on every row. Smoke 2026-09-14 after top-up: searched, served by fireworks, about $0.011 a pair.",
   "smoke_cost_per_pair": 0.011
  },
  {
   "model_key": "meta-small",
   "lab": "meta",
   "provider": "bedrock",
   "model": "meta/llama-4-maverick",
   "generation": "small",
   "tier": "standard",
   "transport": "gateway",
   "endpoint": "https://ai-gateway.vercel.sh/v1/chat/completions",
   "search": "vercel:perplexity_search",
   "input_price_per_m": 0.5,
   "output_price_per_m": 1.5,
   "search_price_per_call": 0.005,
   "note": "Meta does not serve Llama itself; Amazon Bedrock is the pinned host. Smoke 2026-09-14: searched, served by bedrock, about $0.011 a pair.",
   "smoke_cost_per_pair": 0.011
  },
  {
   "model_key": "alibaba-small",
   "lab": "alibaba",
   "provider": "alibaba",
   "model": "alibaba/qwen3.7-flash",
   "generation": "small",
   "tier": "standard",
   "transport": "gateway",
   "endpoint": "https://ai-gateway.vercel.sh/v1/chat/completions",
   "search": "vercel:perplexity_search",
   "input_price_per_m": 0.03,
   "output_price_per_m": 0.13,
   "search_price_per_call": 0.005,
   "note": "Smoke 2026-09-14: searched, served by alibaba, about $0.005 a pair.",
   "smoke_cost_per_pair": 0.005
  },
  {
   "model_key": "moonshot-small",
   "lab": "moonshotai",
   "provider": "novita",
   "model": "moonshotai/kimi-k2",
   "generation": "small",
   "tier": "standard",
   "transport": "gateway",
   "endpoint": "https://ai-gateway.vercel.sh/v1/chat/completions",
   "search": "vercel:perplexity_search",
   "input_price_per_m": 0.57,
   "output_price_per_m": 2.3,
   "search_price_per_call": 0.005,
   "note": "Moonshot does not serve Kimi through the gateway itself; Novita is the only host offered. Smoke 2026-09-14: searched, served by novita, about $0.008 a pair.",
   "smoke_cost_per_pair": 0.008
  },
  {
   "model_key": "zai-small",
   "lab": "zai",
   "provider": "zai",
   "model": "zai/glm-4.7-flashx",
   "generation": "small",
   "tier": "standard",
   "transport": "gateway",
   "endpoint": "https://ai-gateway.vercel.sh/v1/chat/completions",
   "search": "vercel:perplexity_search",
   "input_price_per_m": 0.06,
   "output_price_per_m": 0.4,
   "search_price_per_call": 0.005,
   "note": "Smoke 2026-09-14: gateway refused on free credits (rate-limited until paid credits are added); retry after top-up. Smoke 2026-09-14 after top-up: searched, served by zai, about $0.011 a pair.",
   "smoke_cost_per_pair": 0.011
  },
  {
   "model_key": "minimax-small",
   "lab": "minimax",
   "provider": "minimax",
   "model": "minimax/minimax-m2.5",
   "generation": "small",
   "tier": "standard",
   "transport": "gateway",
   "endpoint": "https://ai-gateway.vercel.sh/v1/chat/completions",
   "search": "vercel:perplexity_search",
   "input_price_per_m": 0.3,
   "output_price_per_m": 1.2,
   "search_price_per_call": 0.005,
   "note": "Smoke 2026-09-14: searched, served by minimax, about $0.007 a pair; slow, 156 s on the test prompt.",
   "smoke_cost_per_pair": 0.007
  },
  {
   "model_key": "openai-luna",
   "lab": "openai",
   "model": "gpt-6-luna",
   "generation": "small",
   "tier": "standard",
   "transport": "direct",
   "endpoint": "https://api.openai.com/v1/responses",
   "search": "web_search",
   "input_price_per_m": 0.1,
   "output_price_per_m": 0.5,
   "search_price_per_call": 0.01,
   "note": "GPT-6 Luna, released 2026-09-22 at $0.10 in, $0.50 out per million, OpenAI's high-volume tier. Smoke 2026-09-22 on customer data platforms, six prompts: searched on 6 of 6, 15 searches, about $0.028 a pair, 4.0 sources cited an answer, 28 s; version string gpt-6-luna. Joined the free set for the October 2026 Edition, decided 2026-09-22 on the smoke.",
   "smoke_cost_per_pair": 0.028,
   "since": "2026-10"
  },
  {
   "model_key": "meta-muse-glimmer",
   "lab": "meta",
   "provider": "togetherai",
   "model": "meta/muse-glimmer-30b",
   "generation": "small",
   "tier": "standard",
   "transport": "gateway",
   "endpoint": "https://ai-gateway.vercel.sh/v1/chat/completions",
   "search": "vercel:perplexity_search",
   "input_price_per_m": 0.35,
   "output_price_per_m": 1.5,
   "search_price_per_call": 0.005,
   "note": "Muse Glimmer 30B, Meta's open-weight model distilled from Muse Spark, released 2026-08-09, $0.35 in, $1.50 out per million, served by Together AI on the gateway (Meta does not serve it). Meta's low-cost model on its current line; it runs beside Llama 4 Maverick, the line Meta has left, so one edition holds both. Smoke 2026-09-30 on customer data platforms, mid-market, six prompts: searched on 6 of 6, 3.3 searches an answer, about $0.021 a pair, 13.7 sources cited an answer, 44 s; version string meta/muse-glimmer-30b via togetherai. Joined the standard tier for the October 2026 Edition, decided 2026-09-30 on the smoke.",
   "smoke_cost_per_pair": 0.021,
   "since": "2026-10"
  }
 ],
 "settings": {
  "temperature": "provider default",
  "max_output_tokens": 4000,
  "anthropic_max_search_uses": 5,
  "fresh_session_per_prompt": true,
  "system_prompt": "none"
 },
 "pricing_note": "List prices captured 2026-09-08 from each provider pricing page. Gemini prices rise on 2027-01-01. Gemini grounding has 5000 free requests per month, not modeled. Perplexity request fee assumes medium search context unless the response reports usage.cost.total_cost. Free-tier list prices captured 2026-09-14 from each lab pricing page (direct) and the Vercel AI Gateway model list (gateway), which carries no markup.",
 "gateway": {
  "endpoint": "https://ai-gateway.vercel.sh/v1/chat/completions",
  "search_tool": "vercel:perplexity_search",
  "search_price_per_call": 0.005,
  "note": "Models without a native search tool are asked through Vercel AI Gateway in the OpenAI Chat Completions format, pinned to the lab own provider or to a short named list of hosts in order (providerOptions.gateway.only and order), no model fallback. search: client means the search tool is a function the collector runs itself against Perplexity's search API (the index the gateway's own tool reads), so the results each model was shown are recorded as its source list, the way the direct models' citations are; the September 2026 standard run used the gateway's server-side tool, which returns no source list. Zero markup on list prices; a search is $5 per 1,000.",
  "search": "client",
  "search_endpoint": "https://api.perplexity.ai/search",
  "search_results": 5
 },
 "judges": {
  "student": {
   "model": "accounts/ai-indexes/models/ai-indexes-judge-qwen3-14b-run5#accounts/ai-indexes/deployments/to5jxbcl",
   "display": "ai-indexes-judge-qwen3-14b",
   "transport": "openai",
   "endpoint": "https://api.fireworks.ai/inference/v1/chat/completions",
   "key_env": "FIREWORKS_API_KEY",
   "input_price_per_m": 0,
   "output_price_per_m": 0,
   "note": "The distilled judge: Qwen3 14B with the run 5 LoRA merged (reports/judge-distill/run-5; Hugging Face AI-Indexes/ai-indexes-judge-qwen3-14b-lora tag run-5), served on a Fireworks on-demand deployment (one H100, BF16, scaling to zero after 5 minutes idle), temperature 0, thinking off, the rubric and schema of the judge above with the run 5 no-first-choice sentence (system_suffix). Measured before use: the same leader as Opus in 52 of 52 categories of the October calibration repeat, largest share gap median 1.1 points, at most 3.1 (reports/judge-distill/run-5/parity); the Fireworks deployment and the pod gave the same first choices on 526 of 541 settled answers. Billed by GPU time while awake, not by token, so a run should collect with --defer-judge and judge in one --rejudge pass. Reads the categories that name it (categories.json \"judge\": \"student\").",
   "system_suffix": "No first choice is a correct reading: if the answer lists or compares options as the best or top ones for the buyer without singling one out, label none of them first_choice; each is alternative, or mention when it is only named or compared. Label first_choice only a product the answer itself picks for the asked scenario.",
   "defer": true
  }
 }
}