Deploying AI safely inside a regulated business

Most published advice on "AI in the enterprise" is written from the vantage point of the vendor. It assumes deployment friction is a cultural or budget problem. Sitting on the other side — inside a regulated group where the output of an AI system has to satisfy a regulator, not just a stakeholder — the friction turns out to be something else. It is that a great deal of general-purpose AI advice quietly fails the regulator's questions.

A short set of lessons from that work, applicable well beyond pharmaceuticals.

The first regulator question is not "how does it work"

It is "how would you show me it worked, in this specific case, on this specific document?" A model that is 95% accurate on a benchmark is not answering that question. What is answering it is a system where every output can be traced back to a specific input, a specific version of a prompt, a specific model artefact, a specific set of retrieved sources, and a specific human who reviewed it before it left the organisation.

The corollary is that a lot of what feels like "productionising" an AI system — reliability, latency, cost — matters much less than a proper record of what happened. If the record is good, everything else can be re-run. If the record is thin, you cannot even answer the question of what went wrong.

Responsible AI is not a policy document

Every regulated company has a responsible-AI policy. Almost none of them survive contact with a real question. What survives contact is a technical architecture in which the responsible-AI properties are enforced by construction: sources are attributable because retrieval is done inside a controlled corpus; outputs are attributable because they are logged with their inputs; humans are in the loop because the workflow makes them the last step before anything is submitted.

The failure mode of responsible-AI-as-policy is not that people ignore the policy. It is that a well-meaning team meets the letter of the policy while shipping a system that would still embarrass the company on a bad day. Enforce the properties in the architecture, and the policy becomes documentation of what the system already does.

Human oversight is not a review — it is a workflow

"Human in the loop" has become weakened by overuse. In a regulated context it means something specific: the human is the accountable actor for the output, and the workflow gives them everything they need to be so — the draft, the sources, the model's own uncertainty where relevant, and time. A workflow that produces a fluent draft with sources buried behind a link, on a two-hour turnaround, is not human-in-the-loop. It is human-approves-blind.

The workflow test is a simple one. Ask the reviewer, after the fact, how they would defend a specific claim in the output. If the answer is "the AI produced it and I checked it looked right," the workflow is broken, whatever the policy says.

Auditability compounds where accuracy does not

The counter-intuitive lesson: in a regulated setting, an AI system's value grows more from its audit trail than from its accuracy. A slightly less accurate system with a perfect audit trail is more useful to a signatory than a more accurate one with a thin trail — because the signatory can defend the first, and cannot defend the second. Investment in retrieval, source binding, versioning and logging pays back for a very long time. Investment in an extra point of accuracy fades as the underlying models improve.

This is why we built facilit8 the way we did. Every claim in an output is bound to its exact source. Every model artefact and prompt version is recorded. Every human sign-off is captured. Not because it is virtuous — because it is what makes the output defensible when someone asks.

What follows for a regulated buyer

Three practical things to ask of any AI system before it goes near a filing:

• Can I trace this output to its source? Not "was the source retrieved" — can I click through to the exact document, revision and passage that supports the claim?
• Can I reproduce this run? If a regulator asks in six months how a specific line was produced, can we replay it — same inputs, same prompt, same model, same sources — and get the same output?
• Is my accountable person actually in the loop? Not in name. In the workflow. With enough visibility to defend the sign-off if asked.

If the answer to any of these is "not really," the system is not ready to touch a regulator-facing artefact — however clever it is.