The Router Is the Interesting Part

The question that exposes most prior-auth automation designs surfaces during security review: which model reads the chart? Not the model — which one, at which step. The step that parses a fax and the step that drafts a denial rationale are different jobs, even when the architecture discussion has treated the workflow as though it uses exactly one model.

On August 11 NVIDIA published Nemotron 3.5 Lightning and NeMo Switchyard. The model is a 30-billion-parameter mixture-of-experts with roughly three billion parameters active per token — built for always-on agents doing high-volume specialized work — that NVIDIA says delivers up to 4x faster output and 30% faster agentic task completion against others in its class. That will get the coverage. It is not the interesting part.

Switchyard is. It is an open-source routing library that sends each step of an agent workflow to the most capable and efficient model for that step, and NVIDIA reports it holds frontier-level accuracy while cutting task completion cost to nearly one-third of using Opus 4.8 alone. Read that as an engineering result and it is a cost story. Read it as an operator and it is something else: the assumption that a workflow runs on a model has just stopped being true, and it stopped being true in a library anyone can adopt this week.

One workflow, several trust boundaries

Here is what changes for a health plan, and it has nothing to do with tokens.

A prior authorization workflow has maybe nine steps. Pull eligibility. Read the clinical note. Find the applicable policy version. Assemble the criteria evidence. Draft a rationale. Route to a nurse. Escalate to a medical director. Generate the notice. File the record. Before routing, those nine steps had one data-egress posture, because they all ran against one endpoint. You approved it once.

After routing, step two might run on a 30B model sitting on hardware in your own data centre, and step five might run on a frontier API in someone else's. Both inside one workflow, both under one workflow name, and the difference between them is a config line that an engineer can change without telling anyone. The router did not create a new risk. It made an old one granular, which is worse, because governance that was written at the workflow level no longer describes what the workflow does.

This is the part that a cost framing misses entirely. Nobody is going to reject routing — a two-thirds reduction in task cost is not a thing an operations executive declines. It will arrive whether or not anyone has decided how to govern it.

The obvious objection, and why it does not hold

The reasonable pushback is that this is a solved problem: you allow-list the endpoints, you put a proxy in front, and routing becomes an infrastructure detail like a load balancer. Route wherever you like, nothing leaves the boundary you drew.

That works right up until the routing decision becomes content-dependent, which is the entire point of Switchyard. A load balancer does not read the payload. A model router does — that is how it knows a step is hard enough to need a bigger model. So the router is making a data-dependent choice about where data goes, and the input to that choice is the same clinical text you are trying to govern. An allow-list can tell you the set of places a request might land. It cannot tell you why this request landed there, and "why" is the whole of an audit.

Which means the control has to sit at the step, not at the perimeter. Each step needs to declare what class of data it handles and what class of model may see it, and the router has to be constrained by that declaration rather than free to optimise across it. Cost is allowed to pick between eligible models. It is not allowed to define eligibility.

What routing makes true that was previously a preference

We have been saying for a while that a foundation model should be a replaceable component rather than an architecture, and routing attaches operating economics to that claim. When eligible routine steps can run on an efficient model and harder steps can reach a more capable one, model portability becomes a measurable operating decision rather than a design preference.

It also changes what a private model is for. The argument for hosting your own has been data residency, and residency alone has never been quite enough to win — plenty of plans decided a BAA and a good contract covered it. A self-hosted model can create a predictable low-cost route under enterprise control, and NVIDIA describes Nemotron as a customizable open model that can be post-trained on an organization’s own domain data. If evaluation on the plan’s approved cases shows that a domain-tuned model meets the quality threshold, it can become the default path while frontier models remain available for harder cases.

That is a real inversion. The privately-hosted model stops being the compliance concession and becomes the default route, with the frontier model kept for the cases that earn it.

What we would ask before turning a router on

Four questions, and they are not rhetorical — we ask them of our own workflows.

• Which steps in this workflow are permitted to reach a model outside our boundary, and where is that declared? If the answer is "the router config", the answer is no.
• When a step is routed, is the model that actually served it recorded in the run, or only the model that was requested? A record that names the intended model and not the serving one cannot reconstruct anything.
• Can the routing policy be changed without a review? If yes, it is not a policy.
• What happens when the preferred model is unavailable — does the step fail, or does it quietly fall back to something with a different data posture?

The last one is the one that bites. Fallback logic is written by engineers on the day something breaks, and a fallback that crosses a boundary is indistinguishable from working software right up until somebody reads the logs.

The part that stays the same

None of this changes the accountability boundary around a coverage decision. An authorized licensed physician or health care professional still makes the medical-necessity determination. The model — whichever one served the step — assembles and proposes. Lower model cost does not justify a lighter control gate; routing changes model selection, not accountability.

What routing does change is the honesty requirement. A year ago you could describe your AI posture with one sentence about one endpoint. That sentence is now false in most deployments, and the plans that notice first will be the ones that wrote their controls at the step rather than at the perimeter.