Before You Rely on an AI Workflow

Buying is not relying

There is no shortage of AI procurement guidance. Ask about certifications, data handling, model change control, audit trails, termination terms. It is sensible, it is converging, and it answers a real question: is this vendor safe to buy?

Buying is not relying. The question that produces expensive surprises is different: is this workflow — the one we already run, assembled from tools that each passed procurement — safe to lean on harder than we currently do?

Nothing in a purchase process examines the assembly. Every component can be individually defensible while the thing you actually operate is not.

Where reliance actually fails

The failures cluster at the seams, not inside the components.

• Dropped assumptions at handoffs. A step produces output under conditions the next step does not know about. Nothing errors. The assumption simply stops being true somewhere downstream.
• Checks that quietly retired. A human used to review step three. The review stopped when the tool got good, and nobody recorded that the review had been load-bearing until something got through.
• Confidence that means less than it looks like. A score, a citation, a green status. Each was designed to answer a narrower question than the one it is now being read as answering.
• Drift. Inputs change shape, volume, or provenance. The workflow's checks were calibrated to what it used to receive.
• False comfort from recency. "It has worked for months" is evidence about the conditions of those months. Reliance is a claim about conditions you have not seen yet.

Questions worth asking before you lean harder

1. What would we be unable to reconstruct? Take one recent output. Can someone establish what it was based on, what was assumed, and under whose authority it went out — without forensic effort? If not, the reliance is already ahead of the evidence.
2. Which human check was retired, and what replaced it? Retiring a check is usually correct. Retiring it without noticing it existed is the problem.
3. Where does this workflow say "I don't know"? A process that never returns uncertainty is not more reliable; it is less legible. Something that cannot report its own limits has told you nothing about them.
4. What is the failure we would notice last? Loud failures are cheap. The costly one is the plausible wrong answer that flows downstream and gets acted on.
5. What can be checked rather than believed? This is the general form of the other four. Where the honest answer is "we would have to take its word for it," you are relying on assertion — which may be fine, as long as it is a decision rather than an assumption.

Why this is a different object from a readiness assessment

These questions concern operational reliance: whether a workflow a team already runs is in a state to be leaned on. That is distinct from agent readiness, which concerns whether your public and operational surfaces can be found, understood, trusted, and acted on by external AI agents.

Both matter and they get conflated constantly, usually because both arrive labelled "assessment." One looks inward at a process you operate; the other looks outward at a surface others act on. The category boundary is drawn more fully on agent readiness is not a generic AI audit.

Where this sits

Every question above is a version of one question: can this be checked, or must it be believed? That is what governable AI action under human authority is for — AI acting under your name with legibility, bounded delegation, reviewable memory, and inspectable action, so that reliance rests on something examinable. For the larger object, see the Verse.