Safety · Deployment
A model can be excellent in evaluation and still fail a customer: a provider times out, a tool call leaks, a search budget runs dry. Deployment safety is the set of controls between a good model and a good answer.
Controls
Every answer streams through a guard. Text that opens like machine output is held until it is clearly prose. A leaked tool call is never shown: the route retries once with grounded search, then falls back to a plain recovery message. Each leak is logged as an event.
Requests route through an ordered list of reviewed providers under a cost policy. A failing provider is skipped, not waited on. The catalogue is served from a reviewed snapshot and refreshed in the background.
Live search has a fixed time budget per request. When it is spent, the assistant answers from what it knows and flags staleness in one line rather than hanging.
Connected-app actions that send, change or delete are queued as pending actions. The customer confirms each one. Read-only tools run without a prompt.
Messages are capped in length and history is windowed, which bounds cost per turn and removes a class of abuse.
If no model, search or agent runtime is configured the route returns a clear error instead of an empty stream.
Monitoring
A reliability job probes the product the way a customer uses it and raises incidents to the on-call owner.
| Probe | What it checks | Reads | On failure |
|---|---|---|---|
| Direct chat probe | The provider chain returns a real answer to a fixed prompt | Nothing from customers | Critical incident |
| Live search probe | Grounded search returns within budget | Nothing from customers | Critical incident |
| Agent canary | Specialist runtime completes a turn for a synthetic customer with no profile row, so it only reads and never writes | Nothing from customers | High incident; customers still served by the direct chain |
| Real answer rate | User turns in the last hour that received no assistant reply within 3 minutes | Timestamps and roles only, never content | Critical incident at 2+ unanswered and 25%+ of judged turns |
Release process
The test suite asserts that the chat stream loop still passes through the answer guard; a refactor that bypasses it fails the build.
What can this send or delete? What does the customer see on failure? What alert fires? How is it rolled back?
Production deploys are immutable; the previous deployment stays promotable. The five-minute probes run against the new build immediately.
When a probe fails after a deploy, the previous build is promoted first. Diagnosis happens on the rolled-back incident, not on live customers.
Impact is counted, never quoted. The write-up ends with a system change, and the research index is updated when the change is material.
Known limits
This page describes controls in production as of October 2026. It is updated when controls change.
The reliability work is published in the research index.
