Why document automation in produce trade was impossible until now

Somebody in your office is retyping a packing list. Forty pallets, six columns each, and the same job again for the next container. It is the least interesting work in the building, and it is also where a wrong box count enters your stock figures.

Nobody chose this. Software has been reading documents since the 1990s, and importers have been asking for it for about as long. The honest answer is that until recently, nothing could read these particular documents. The reasons are specific, and they tell you what to ask a supplier now that something finally can.

What makes a produce document hard #

A packing list for a container of fruit is not a page of prose with a few numbers in it. It is a table of forty to a hundred rows, each carrying a pallet number, a variety, a calibre, a box count, a net weight and usually a grower code. The whole value of the document is in that grid.

Every exporter has a different layout. A machinery importer buys from five suppliers. A produce importer buys from sixty growers across eight origins, and every packhouse prints from its own software. Peru alone can send you a dozen formats. They also move: a new column appears in the middle of the season because a retailer asked for one.

The table often has no lines. Some packing lists are ruled like a spreadsheet. Others are columns of text held in place by spacing alone, or ruled in a way that groups three rows into one visual block.

Calibres migrate into the headings. Many packhouses do not give calibre a column of its own. They put the calibres across the top — 16, 18, 20, 22 — and the box counts sit underneath in a grid. Reading that means understanding that the number 24 in the heading is a size, while the number 24 in the cell below it is a quantity.

The words are in four languages. Cajas, colli, bultos, doosen, C/S — all of them box counts. Peso neto, peso líquido, nettogewicht. And one document frequently uses two languages at once: a Peruvian asparagus packing list may head one column “Esparrago” and the next one “Asparagus”.

The file is often a photograph of a printout. The exporter has the data in a spreadsheet. What reaches you is a PDF printed from it. Sometimes it is a scan of that PDF, or a phone photograph of the scan taken at the packhouse at six in the morning. Each step throws away structure and adds skew, shadow and a stamp across the middle.

And it is frequently not one document. One attachment, six pages: the bill of lading, the invoice, the packing list, the phytosanitary certificate. No separator page, no titles, four different layouts.

What legacy OCR actually did #

OCR means optical character recognition, and the name is exact about the scope. It turns pixels into characters. It does not know that the third column holds box counts, or that this page is a packing list at all. Everything above the character level had to be built on top, and two ways of building it are still being sold today.

Zonal OCR, also called template OCR. You take one sample document, draw a rectangle around the box count, and tell the system that the box count lives there. It works — on that template it works very well. It stops working on the second exporter’s layout, and it stops working on the first exporter’s layout the day they add a column. The economics only make sense when one template covers thousands of documents. That is true for a bank processing its own forms. It is false for an importer with sixty growers, one of whom ships four containers a season.

Table detection from ruling lines. Find the horizontal and vertical lines on the page, then treat the boxes between them as cells. This is elegant when the lines exist and sit where the data sits. On produce packing lists they often do neither.

Both share a deeper problem, and it is the one that stopped this whole category working here. Ninety-five per cent accuracy is worth nothing on this document. A page of prose read at 95% is readable. A packing list read at 95% has roughly a dozen wrong cells in it, in unknown positions, and the totals no longer add up. Somebody then checks all of it — which is the work you were trying to remove.

So the honest history is not that OCR was almost good enough. OCR solved a real problem: turning a page of text into text. A packing list is not a page of text. It is a spreadsheet that was printed, and the printing destroyed the structure that mattered.

What changed #

Modern vision models read an image and language together. They were trained on documents among everything else, so a table, a heading and a stamp are all things they have seen. They read a page roughly the way a person does: position, layout, headings and the meaning of the words, all at the same time.

Three consequences follow, and each one removes a failure from the list above.

A layout it has never seen is not a special case. This is the whole difference, and everything else is detail. There is no sample to collect, no template to configure, no rectangle to draw. A new grower’s first document is read as well as the sixtieth from an old one.

The words are understood rather than matched. Cajas is a box count because the model knows Spanish and knows what a packing list is — not because somebody added it to a list of synonyms. Two languages on one page is not a setting anyone has to switch on.

Damage can be read around. A stamp over a number, a skewed scan, a handwritten correction in the margin. A model that knows what a packing list row looks like can use the rest of the row. If the box weight is 4.5 kg and the net weight is 1,350 kg, a smudged box count of 300 is recoverable. Character recognition had no way to do that, because it never had a row — only pixels.

The same shift covers the work around the reading. Deciding that a file is a phytosanitary certificate rather than an invoice used to need a rule for each issuer. It is now a judgement made from the content. A six-page merged PDF can be split at the boundaries between the documents inside it, because “this page starts a new document” is something a reader can see.

What it still gets wrong #

This is the part the demonstrations leave out, and it decides whether any of this is safe to run on your consignments.

The failure mode changed, and it got quieter. Old OCR produced rubbish that looked like rubbish: 8O0, l23, half a word. You could see it on the screen. A vision model produces 800 where the document says 300 — right format, right field, plausible number, nothing to notice. That is a more dangerous error, not a smaller one.

The same document read twice can give two answers. On a dense sixty-row table these models are not consistent with themselves. Read it again and a few cells may differ. This is real, it is well known, and nobody has fixed it.

It does not know your business. It does not know which grower code belongs to which of your suppliers, or whether “Cat I” is acceptable under this contract. It does not know which of two conflicting total rows your customs broker uses. None of that is written in the document.

And a spreadsheet can hold more than it shows. An Excel packing list often has hidden columns, and they are usually hidden for a reason. They carry the previous customer’s order, or a working column the exporter forgot about. A model given the raw file reads them, because they are in the file. No better model fixes this. The fix is to remove what the sender hid before the sheet reaches the model at all.

Why the arithmetic matters more than the model #

Given all of that, why is any of it safe to run without somebody watching each document?

Because the document tells you whether it was read correctly. A packing list declares its own totals: number of pallets, total boxes, total net weight. Those totals are a check on every row above them, and the exporter wrote them long before anyone tried to automate anything. If the extracted rows add up to the declared box count, and the weights reconcile, the numbers are almost certainly right. If they do not, something is wrong and you know it in the same second.

That check is not clever and it is not new. It is arithmetic. But it turns the dangerous failure — a confident wrong answer — into a visible one, and that is the difference between a demonstration and a production system. It is also why the operator’s job changes shape rather than disappearing. Instead of typing two hundred and forty cells, they look at the one consignment where the boxes did not add up.

A second attempt helps as well. Where the first one fails the arithmetic, reading the document again with a different model costs a few seconds and rescues a fair proportion of documents. The condition is what matters: the second answer is kept because it makes the totals reconcile, not because the second model is trusted more than the first.

Which documents stop a container, and when each one has to arrive, is a separate subject — understanding the documents in a fresh produce import covers it. The connection to this page is simple: a document nobody has read is a document nobody has checked. The bill for that arrives as demurrage two weeks later.

What to do with this #

Ask for the file, not a picture of the file. The exporter has a spreadsheet. If they send the spreadsheet instead of a PDF printed from it, every calibre and every weight arrives as a number rather than as ink. It is a two-minute conversation and it is the single largest improvement available to you.

Stop treating reconciliation as clerical work. Checking that the rows add up to the declared totals is not tidying. It is the control that makes an automated read of the document trustworthy, and it is the one step that should never be skipped for speed.

Test any supplier the honest way. Give them ten documents from ten different growers, including your two worst scans, and give them none of it in advance. If the answer is “send us fifty samples per exporter and we will train on them”, that is template OCR under a newer name. It will break the first time a packhouse adds a column. A system that can read a format it has never seen does not need your samples. That is the whole test, and it takes an afternoon.

This is what we do with the documents that reach your inbox. Each one is classified, split when several arrived as a single PDF, and matched to the shipment it belongs to before anything is read out of it. Receiving documents by email has the mechanics.

None of this makes the paperwork disappear. A container of grapes still arrives with the same stack of documents it arrived with ten years ago. What has changed is that reading them is no longer a job done one cell at a time, by somebody who would rather be doing something else.

FAQ #

Can AI read a scanned packing list? #

Usually yes, including a photograph taken on a phone. Modern vision models handle skew, shadow and moderate blur, and they can often work around a stamp printed over a number. What they cannot do is read a figure that is genuinely not there. A cell completely covered by a stamp is lost, and the honest answer is to flag that document for a human rather than guess the number.

Is this just OCR with a new name? #

No, and the difference is testable. Character recognition turns pixels into characters, and everything above that — which column is which, what the words mean — had to be configured per template. A vision model reads layout and meaning together, so a document format it has never seen needs no setup. If a supplier asks for sample documents to configure or train on, you are being offered the old technology.

What happens when it reads a number wrong? #

That is why the totals matter. A packing list declares its own box count and net weight, so extracted rows that do not add up to the declared totals tell you immediately that something was misread. A well-built system rejects those numbers and asks a person, rather than showing you figures it could not verify.

Do I still need someone checking the documents? #

Yes, but the job is different. Instead of reading every packing list, they look at the ones that failed a reconciliation check — typically a small minority. The value is not that the person disappears. It is that their attention goes to the ten per cent of consignments where something is genuinely wrong.

How many document templates does it need to be trained on? #

None. That is the change this whole page is about, and it is the sharpest question to ask a supplier. A system built this way reads a new exporter’s first document without having seen anything from them before.