Auditable Document Workflow

The problem

A small foundation pays school fees for children in Kenya. The paper trail is exactly what you’d expect: photographed receipts of varying quality — handwritten amounts, inconsistent formats, different schools, different terms — arriving as JPEGs and phone-camera PDFs. Behind them, a simple but unforgiving requirement: every payment must be matched against the expected term fee, tracked per child per year, and defensible to anyone who asks where the money went.

The naive AI solution is obvious: point a vision model at the receipts and let it fill a spreadsheet. It’s also exactly wrong. Extraction models are confidently approximate — they will read a smudged 4 as a 9 and move on without a flicker of doubt. In most demos that’s a rounding error. Here it’s a child’s school year recorded as unpaid, or worse, a discrepancy nobody notices until it can’t be reconstructed.

The problem was never “can AI read a receipt?” It can, mostly. The problem was: how do you build a pipeline where mostly is never allowed to silently become recorded?

The system

The design principle is separation of extraction from truth. The model proposes; the workflow disposes.

The pipeline runs in four stages. Extraction: each receipt image is processed into structured fields — school, child, amount, term, date, receipt number — with the model required to flag anything it isn’t certain about rather than guess. Verification: extracted amounts are checked against the known fee schedule for that school and term. A match is a candidate record; a mismatch is a named exception, never a quietly adjusted number. Human confirmation: exceptions and low-confidence extractions are surfaced for a person to resolve against the original image. Nothing enters the tracker without either passing verification or being explicitly confirmed. Recording: confirmed entries land in an annual tracker that maintains payment status per child — with the source image referenced, so any row can be traced back to the paper it came from.

Three rules shaped everything:

Uncertainty must be expensive to hide and cheap to express. The extraction step is explicitly instructed that “I can’t read this” is a valid, welcome output. The moment a pipeline punishes uncertainty, models stop reporting it — and unreported uncertainty is how bad data gets laundered into official records.

Verification against independent ground truth, not self-consistency. The fee schedule exists outside the model. Checking extraction against what the amount should be catches an entire class of errors that no amount of “read it again carefully” ever will.

The audit trail is the product. The spreadsheet isn’t the deliverable; the traceability is. Every recorded number answers three questions on demand: what image did this come from, what rule validated it, and who confirmed it if a human touched it.

What I learned

Small stakes, same physics. This is a tiny nonprofit workflow, but the constraint profile — real money, zero tolerance for silent errors, external accountability — is identical to enterprise compliance. It turned out to be the perfect scale to learn the discipline: every design decision that made this trustworthy is one I’d make in a regulated financial pipeline, just with more zeros involved.

Trustworthy AI is mostly workflow, not model. The extraction model is the least interesting component and the easiest to swap. The value lives in the scaffolding around it: the fee-schedule check, the exception path, the confirmation gate. If I deleted the model tomorrow, the workflow design would survive; the reverse is not true.

Confidence thresholds are a policy decision, not a technical one. Deciding what the system may record on its own versus what a human must see is the single most consequential choice in the design — and it has nothing to do with prompting. Someone has to own that line. Making it explicit, visible, and adjustable is what separates a system from a demo.

Boring is the goal. The finished pipeline is unglamorous: extract, check, escalate, record. That’s the point. In environments where errors compound quietly, the highest compliment a system can earn is that nothing about it is surprising.