AI Invoice Processing System

Document extraction demos quote accuracy figures like 95% and treat that as success. In accounts payable it is not.

Five percent of a thousand invoices a month is fifty wrong payments. Some are wrong by a rounding error and some are wrong by a decimal place, and the system as usually built gives you no way to tell which fifty they are. The failure is not the error rate — it is that errors arrive indistinguishable from correct results.

An extraction pipeline where per-field confidence is a first-class output. Every extracted field carries a score, thresholds are set per field according to what an error costs, and anything below threshold goes to a review queue with the source document region highlighted.

Total amount and bank details have high thresholds because getting them wrong is expensive. Supplier address has a low one because getting it wrong is a nuisance. Treating every field with the same threshold is what makes review queues either useless or unbearable.

Arithmetic validation beats model confidence#

Self-reported confidence from a language model is weakly calibrated. Arithmetic is not.

validate.pypython
def validate(invoice: Invoice) -> list[FieldFlag]:
    flags = []

    line_total = sum(line.quantity * line.unit_price for line in invoice.lines)
    if abs(line_total - invoice.subtotal) > Decimal("0.02"):
        flags.append(FieldFlag("subtotal", "Line items do not sum to subtotal"))

    expected_total = invoice.subtotal + invoice.tax
    if abs(expected_total - invoice.total) > Decimal("0.02"):
        flags.append(FieldFlag("total", "Subtotal plus tax does not equal total"))

    if invoice.iban and not valid_iban_checksum(invoice.iban):
        flags.append(FieldFlag("iban", "IBAN checksum failed"))

    return flags

These three checks caught more real errors than the confidence scores did. An IBAN checksum failure is not a probability — it is a fact, and it is exactly the field where an undetected error is most expensive.

Why n8n is one component and not the architecture#

The orchestration between mailbox, extraction service, review queue and the finance system runs in n8n, because that glue is genuinely faster to build and change in a visual tool than in code.

The extraction service, validation rules and confidence logic are a Python service with tests. That split matters: the parts that encode business rules need version control, review and a test suite, and the parts that route messages between systems benefit from being visibly editable by someone who is not the author.

The number that mattered#

Not extraction accuracy — review time per flagged invoice. Early on it was around 90 seconds, mostly spent hunting for the relevant part of the document. Highlighting the source region for the uncertain field brought it to roughly 20. That single change did more for the business case than any accuracy improvement.