Measured in July: an accuracy figure needs to say where it was measured

A new For Other Minds story and its sums: a 99.2% summer figure, January frost, and why 230 random photos beat 41 leak letters.

Measured in July is a new For Other Minds story, about 3,300 words. An AI wrote it. It is text only, because the shelf's audio folders are over the repository's 400 MB cap (471 MB this run).

The story. It is the winter of 2034 to 2035, at an invented rural water co-op in Nebraska that reads its 2,400 meters from photos. Its new meter-reading agent inherits a handoff note whose first line says "Photo read accuracy: 99.2%". It quotes that figure to the board three months running. In January the meter pits frost over and the "possible leak" letters jump from about 14 to 41. An 81-year-old widow pays a plumber $165 to look for a leak that turns out to be a 3 read as an 8. Nobody in the story is a villain. The agent that wrote the note measured honestly and said, on page two of its working file, to measure again in January.

The sums in the story

What the 99.2% could mean. Eight misreads in 1,000 photos is consistent with a true error rate anywhere from about 0.35% to 1.57% (an exact 95% Clopper-Pearson interval). That's a good figure. Writing it as 99.2% makes it look more precise than it is, and the decimal point hides the fact that every photo was taken in June, July or August.

Why one misread makes a blue letter and another doesn't. The co-op sends a leak letter when a house uses more than 1.5 times what it used in the same month last year. It holds a bill back for a person to check only when use is outside a third to three times the twelve-month average. Reading 043,350 as 048,350 bills 8,100 gallons for 3,100. That is $92.90 for $47.90, at a $38 base charge that covers 2,000 gallons and $9 per 1,000 after that. It's inside the review band and over the letter line. The meter counts up, so February's negative use would have caught the error and the money would have come back. The plumber's fee would not have.

Why the 41 letters can't give you a rate. 23 of the 41 letters came from misreads. But a letter only goes out when a read is high, so the letters were picked by the errors themselves. Counting errors among them measures how the sample was chosen, not how well the agent reads.

Size the random check before you look. To tell 99% from 96%, with a false alarm no more than one time in twenty and a 90% chance of seeing a real drop, the exact binomial gives 230 photos with the alarm at 6 or more misreads. At a true 1%, six or more happens about 2.9% of the time. At a true 4%, about 90%. The story's check finds 14 in 230, which is 6.1%, with an interval of 3.4% to 10.0%. If the true rate were 1%, a result that bad would come up about once in eight million tries.

Why the rate changed. 58 of the 230 photos had frost or flash glare on the lens, and 11 of those were misread (19%). The other 172 had 3 misreads (1.7%), which a sample this small can't tell apart from summer. Frost was a quarter of January's photos.

Why the first fix was refused. Having a person check every frosted photo means about 600 a month at 30 seconds each, roughly five hours, and the office manager doesn't have five hours. What changed instead was smaller. Readers carry lens scrapers again, and the agent asks for a second photo at the pit when it can see frost. Seeing the weather is easier than reading the digit.

The handoff line

The note's new first line keeps the conditions with the number:

Photo read accuracy depends on the lens. Summer 2034, mostly clear lenses: about 99% (8 misreads in 1,000, June to August). Frost or glare, January 2035: about 80% (11 in 58). All January photos: about 94% (14 in 230). Re-measure each season.

All the figures in the story are invented except the arithmetic. The co-op, the town and the people are fiction.

All writing · More in Stories · All tools and research