What is Deterministic vs Probabilistic Behavior?
Also called non-deterministic agents, reproducibility.
Deterministic behavior produces the same output for the same input every time, while probabilistic behavior samples from a distribution and may differ between runs. Language models are probabilistic by default, so agents built on them are not reproducible in the way ordinary software is. Reliable systems place deterministic code around probabilistic decisions rather than trying to remove the variability.
Variation has several sources. Sampling settings such as temperature introduce randomness deliberately. Even at the lowest setting, floating point non-associativity, batching on shared infrastructure, and provider-side model updates mean identical requests can still diverge. Treating a single successful run as proof that a prompt works is therefore unsound, which is why agent evaluation runs the same case repeatedly.
The practical consequence is where to put each responsibility. Anything with consequences, such as permission checks, spending limits, ordering, retries, and validation, belongs in code that behaves identically every time. Interpretation, drafting, and judgment can live with the model. Systems that ask a model to enforce their own rules inherit the variability into the rules themselves.
Testing changes accordingly. Exact output matching mostly does not work. Teams assert on structure and constraints, run each case several times and measure a pass rate, use a separate model as a grader with human spot checks, and record trajectories so a regression can be compared step by step rather than string by string.
A common misconception is that setting temperature to zero makes an agent deterministic. It reduces variability, often substantially, but neither guarantees identical output nor makes the system reproducible across provider versions. The realistic goal is bounded behavior: outcomes that vary in wording while staying inside limits the surrounding code enforces.
Key points
- Outputs can differ between identical runs, even at low temperature.
- Put consequential rules in code and judgment in the model.
- Test with repeated runs and pass rates, not exact matching.
- Provider-side model updates can change behavior with no code change.
In practice
The same agent is asked twice to summarize a contract. One run lists five key clauses, the other lists four and merges two of them. Both are defensible. If a downstream step required exactly five bullets, it would fail intermittently and look like a mystery bug. Requiring a structured field with a validated shape, checked in code, removes the whole class of problem.