A receipt that says 6.008 almost always means 6.000: 600 more photos, two fixes, one dead end

600 unseen CORD receipt photos read 61% of totals. A rule for totals ending 00 and a digit, and a fix to how the page writes a total back: page 58.5% to 61.2%.

ai-bookkeeper reads a receipt photo four times with tesseract and lets the receipt's own arithmetic pick the total. The last run found that our photo rules had been tuned to the same 100 validation photos, and moved development to CORD's train photos. This run read the remaining 600 train photos, none seen before.

The unseen rate holds. Through the command line, 356 of 583 labelled totals came out right (61.1%). Shard 0 read 59% and the held-out test set 58%. So about 60% is the honest rate for a phone photo of an Indonesian receipt with a free OCR engine and no model.

Most misses can't be fixed by the parser. For 148 of the 227 misses, the right total's digits are not in any of the four readings. Last time's idea for those photos was to read the bottom of the photo again on its own. We tried it: a reading of the bottom half found the total on 5 of 149 photos, and the bottom third on 4. These totals are lost to blur, folds and camera angle, not to text above them. We dropped the crop.

Two things the parser could fix:

  • A total ending in 00 and one more digit. "Total 6.008", "Total 31.508", "Total 43.006": tesseract read the last 0 as an 8, 6, 9 or 3. Of CORD's 972 labelled totals, 8 end that way and 604 end in 000. When some reading prints the round amount on any line, we now take it and add a note asking for a check. One of the six photos it fixed was different: it really printed "Before Rounding 110,003" above a total of 110,000. A real total of 6,008 stays 6,008 unless a reading shows 6,000.
  • The page undid its own result. When the readings settled on a total that had no line of its own (say, from the arithmetic), the page wrote Total 110000.00 into the receipts box. On a rupiah receipt that reads back as a different amount, so the page fell back to "the readings disagree" and showed the first reading's wrong total. It now writes Total 110,000 when that reads back correctly. That one change gave the page 14 more correct totals on these photos.
beforeafter
600 new train photos, command line356 / 583362
600 new train photos, page341 / 583357
validation photos, page67 / 9870
held-out test photos, command line / page (scored once)55 / 53 of 9556 / 55

Nothing was lost on any set, and the CORD text sets did not change. The held-out gain is small but points the same way as the development gain. The page now trails the command line by 5 totals, not 15.

What's left is mostly OCR: a second engine or a model reading the 148 photos tesseract can't. Data, commands and every changed receipt are in the evaluation note. CORD is Clova AI's dataset (CC BY 4.0). The photos are read at run time and not copied into the repository.

ai-bookkeeper is a bookkeeping aid, not tax or accounting advice. Check any total it flags.

All writing · More in Tools and projects · More on AI Bookkeeper · All tools and research