Model routing for AI agents: stop overpaying GPT-5.2 for simple tasks

Autonomous agents don’t run one prompt. They run loops — sometimes thousands of LLM calls per job. If every call goes to gpt-5.2-pro or claude-opus-4.6, you’re paying $21/MTok for tasks like “extract a date from this text” or “write a log line.” PRYSM routes each call to the right model automatically, without you changing your agent code.

The cost problem in agent loops

A typical research agent flow:

Plan subtasks              → needs reasoning   → claude-sonnet-4.5 ($3/MTok)
Search 10 web pages        → needs realtime    → sonar-pro ($3/MTok)
Summarize each page        → simple extraction → deepseek-v4-flash ($0.10/MTok)
Cross-check facts          → needs reasoning   → claude-sonnet-4.5 ($3/MTok)
Write final report         → creative writing  → gpt-5-mini ($0.25/MTok)

Steps 3 and 5 are 80% of the call volume but need only 2–5% of the cost. Routing them to frontier models wastes budget.

Drop PRYSM into any agent framework

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="auto",
    openai_api_key="prysm_sk_...",
    openai_api_base="https://api.prysm1.com/v1",
)
# LangChain chains, agents, and tools all work unchanged

from crewai import Agent, LLM

prysm_llm = LLM(
    model="openai/auto",
    base_url="https://api.prysm1.com/v1",
    api_key="prysm_sk_...",
)

researcher = Agent(
    role="Senior Researcher",
    llm=prysm_llm,
)

import os, openai

os.environ["OPENAI_BASE_URL"] = "https://api.prysm1.com/v1"
os.environ["OPENAI_API_KEY"] = "prysm_sk_..."

from openai import OpenAI
client = OpenAI()
# agents.Runner, function tools, handoffs — all unchanged

config_list = [{
    "model": "auto",
    "api_key": "prysm_sk_...",
    "base_url": "https://api.prysm1.com/v1",
}]

Inspect routing decisions

Every response carries a prysm.routing extension so your agent can log exactly why each call went where:

from openai import OpenAI
client = OpenAI(api_key="prysm_sk_...", base_url="https://api.prysm1.com/v1")

def agent_step(task: str, history: list[dict]) -> str:
    resp = client.chat.completions.create(
        model="auto",
        messages=history + [{"role": "user", "content": task}],
    )
    routing = resp.model_extra.get("prysm", {}).get("routing", {})
    cost = resp.model_extra.get("prysm", {}).get("cost", {})
    print(f"[{routing.get('model')}] {cost.get('total_usd', 0):.7f} USD — {routing.get('why', '')}")
    return resp.choices[0].message.content

Sample output from a 5-step research agent:

[claude-sonnet-4.5]  0.0014200 USD — Complex reasoning task
[sonar-pro]          0.0031000 USD — Realtime web search required
[deepseek-v4-flash]  0.0000094 USD — Simple summarization
[deepseek-v4-flash]  0.0000088 USD — Simple summarization
[gpt-5-mini]         0.0001750 USD — Writing task, no reasoning required
Total: $0.0047132 (vs. $0.0620000 all-opus)

Pin specific models per agent role

Some tasks warrant explicit model choices. Use model directly or configure a BRAIN.md:

# Explicit pinning — bypass routing for this call
resp = client.chat.completions.create(
    model="claude-opus-4.6",   # always use Opus for this step
    messages=[...],
)

BRAIN.md in your agent's working directory

model: auto
max_cost_per_request: 0.01

rules:
  - when: reasoning
    model: claude-sonnet-4.5
  - when: realtime
    model: sonar-pro
  - when: simple
    model: deepseek-v4-flash

fallback: gpt-5-mini

When a BRAIN.md is present, PRYSM applies your rules before its own routing logic. Team-level policies propagate automatically to every agent in the repo.

Set a hard cost cap per request

For runaway-proof agents, add X-Max-Cost-USD to your OpenAI client’s default headers:

client = OpenAI(
    api_key="prysm_sk_...",
    base_url="https://api.prysm1.com/v1",
    default_headers={"X-Max-Cost-USD": "0.005"},  # $0.005 hard cap per call
)

If the routed model’s estimated cost exceeds the cap, PRYSM auto-downgrades to the cheapest live model that fits — no exception thrown, no user intervention. See AgentGuard for the full safety layer.

​The cost problem in agent loops

​Drop PRYSM into any agent framework

​Inspect routing decisions

​Pin specific models per agent role

​Set a hard cost cap per request

The cost problem in agent loops

Drop PRYSM into any agent framework

Inspect routing decisions

Pin specific models per agent role

Set a hard cost cap per request