What is Shadow Deployment?
Also called shadow mode, dark launch.
A shadow deployment runs a new version of a system on real production traffic while discarding its output, so users continue to receive results from the existing version. It reveals how the candidate behaves on true inputs without exposing anyone to its mistakes. The two sets of outputs are then compared offline.
Traffic is duplicated at some boundary, usually a proxy or the application layer, and sent to both the current version and the candidate. Only the current version's response is returned. The candidate's response is recorded for later comparison, along with its latency, errors, and resource use. This gives realistic performance data and a paired sample of outputs on identical inputs, which is far stronger evidence than comparing two different traffic slices.
The pattern carries a hard constraint: the shadow path must not cause side effects. If the candidate sends an email, charges a card, writes to a shared table, or calls a partner API, the shadow run becomes a second real run. Agents make this acute, because their whole purpose is to act. Practical shadowing for agents requires tool calls to be routed to sandboxes or intercepted and logged instead of executed.
Cost is the other constraint. Shadowing doubles model inference for whatever fraction of traffic is duplicated, so teams typically shadow a sampled percentage rather than everything, and run for a bounded window. Because tokens are commonly metered by usage, a full-traffic shadow of an expensive configuration can quietly become one of the larger line items in a month.
Shadow deployment answers a narrower question than a split test. It tells you what the candidate would have produced and how it would have performed, but not how users would have reacted, since nobody saw the output. It is therefore complementary to a controlled release: shadow first to catch crashes, latency regressions, and obviously wrong outputs, then expose a small share of users to measure response.
Key points
- Candidate runs on real traffic, its output is discarded
- Produces paired outputs on identical inputs for comparison
- Side effects must be sandboxed or the shadow becomes real
- Duplicated inference roughly doubles cost for shadowed traffic
- Cannot measure user reaction, only system behavior
In practice
Before switching the routing model on a triage agent, a team mirrors 10 percent of incoming tickets to the candidate. Its tool layer is swapped for a recorder that logs intended calls without executing them. After two days, 4,100 paired decisions show the candidate agreeing with production on 96 percent of tickets, disagreeing mostly on multilingual ones, and adding roughly 300 milliseconds of latency.