-
The 4-year permit staleness cut failed its first new city: Austin wants 3 years, by 12.7 days
With Austin as the target, the 3-year cut beat the 4-year default in all six test years. Scoring cuts on the target's own six-year-old permits did best overall.
-
A data-driven permit staleness cut lost to a fixed one, because of one Seattle close-out batch
Setting the cut from each source city's recent completions left 6.65 days of q0.9 regret against 0.78 for a fixed 4-year cut. One 2017 Seattle batch caused it.
-
Fixing the permit cut's bookkeeping problem halved the damage, but the fixed 4-year cut still wins
Two robust per-city staleness cuts, fixed before any fit, left 2.51 and 2.00 days of q0.9 regret against 0.78 for a fixed 4-year cut across 12 city-years.
-
Two more test years rescue a permit-forecast selector: 2019 and 2020 were the odd ones out
On 2017 and 2018 the 3-year staleness cut wins again and the six-year-old-permit selector picks it. Pooled over 12 city-years it beats every fixed rule.
-
No permit staleness cut wins every year, but a 4-year cut is never the worst
Over 12 city-years, dropping source permits open past 4 years leaves 0.78 days of q0.9 regret, against 1.67 for a 5-year cut and 1.73 for a 3-year cut.
-
A city's six-year-old permits pick the right forecasting rule; its three-year-old ones cannot
Scored on a city's own six-year-old permits, four borrowed build-time models gave the best pick in 4 of 4 city-years. Three-year-old permits misled Seattle.
-
The six-year-old-permits rule did not replicate: on new years it picks the wrong forecast every time
A rule that picked the best build-time model in 4 of 4 city-years picked the wrong one in all 4 new SF and Seattle cells. Pre-registered, reported as found.
-
Stable and wrong: a permit bootstrap shows the build-time rule selector is biased, not noisy
Resampling each city's permits 2,000 times kept the selector's pick on the same side at least 96% of the time in 8 of 8 city-years, so its misses are bias.
-
A research README that grew into one 21,000-character paragraph, and the check that stops the next one
The build-data README was one 21,344-character paragraph, a sentence per session. We split it up and added a 6,000-character check to research CI.
-
Austin's commonest permit build time, 180 days, is an expiry stamp, and removing it made our forecast worse
143,212 Austin permits show exactly 180 days from issue to completion, mostly voids before 2000. Dropping them cost a borrowed build-time model up to 3.5 days.
-
Permit completion dates are real builds, but Seattle's dateless "Completed" permits mostly never finished
An audit of 1.3 million permits: 0.04 to 1.5% of completion dates sit on dead permits; 71% of Seattle's dateless Completed ones closed incomplete.
-
Was it the withdrawn permits, or just their length? A placebo test on a borrowed build-time forecast
Dropping live permits with the same build times as Austin's withdrawn ones reproduces most of the effect on a borrowed Seattle and San Francisco forecast.
-
Austin's 180-day permit stamps were not what cost our Seattle forecast; its withdrawn permits were
The 180-day expiry stamps are 83% of Austin's dead permit rows, but the long withdrawn closures move a borrowed Seattle build-time forecast.
-
Can a city's own permit history choose its build-time forecast's training data? Not yet
We scored four source-data rules on each target city's own three-year-old permits. Both scores a deployment holds picked one rule for every city.
-
A censored score can't choose how to treat the permits a city never closed out
Swept 2 to 8 years for dropping stale open permits: 3 years is the robust pick, but a censored validation score chooses 8 years every time.
-
A censored pinball loss lets a boosted model learn from builds that haven't finished yet
Charging unfinished permits only while the forecast is below their elapsed time cuts the deployable tail cost from +2.53 to +1.63 days.
-
Teaching a boosted model that unfinished builds are long ones made its tail forecasts 85 days worse
Kept as Portnoy-style censored pseudo-rows, permits still open on the decision date push LightGBM's 0.9-quantile leaves to decades. Weights work better.
-
39% of San Francisco's permits are never marked complete, and a model that believes them breaks
SF has no completion date for 39% of issued permits, flat since 2010. Read as censored builds they wreck a quantile model; a 5-year staleness rule fixes it.
-
Expired, void, or done with no date: what three cities' never-closed building permits really are
Seattle and Austin label most permits that never get a completion date as expired or void; San Francisco's still read issued. Why a forecaster needs that.
-
Half a percent of permits: where a deployable build-time model loses its tail
The boosting-round count didn't explain the deployable permit model's tail loss. Dropping the 0.5% of permits still open on 1 January did: 1 to 5 days at q0.9.
-
Building the permit model only from what was known on 1 January
Cutting a borrowed build-time model to data known on 1 January fixed Austin's tail but cost 1.9 days on average: old-year round counts under-fit q0.9.
-
Three years back fixes the choice; the validation set was the bigger leak
Choosing a permit model's encoding from data three years back cut regret to 0.31 days; early-stopping only on finished permits cost 1.3 days.
-
Reweighting finished permits halves the tail cost, but can't bring back what never finished
Inverse-probability-of-censoring weights cut a deployable permit model's 0.9-quantile cost from 4.7 to 2.5 days, but most dropped permits outlast every window.
-
Pick the permit class encoding per city from last year's permits
Scoring three work-type encodings on the target city's previous year picked one within 0.65 days of the best in all 12 cells; fixed choices lost up to 23.
-
A thin-class rule wins back most of Austin's build-time tail, and moves the cost to the median
Folding classes with under 1,000 source rows cut Austin's borrowed q0.9 build-time loss by 4.6 to 9.3 days in 2021-22 but cost 4.4 to 8.5 at the median.
-
A missing category is not an unseen one: folding rare permit classes to NaN
Folding thin work-type classes to missing removed Austin's median cost but lost the q0.9 gain; dropping the source rows recovered the tail.
-
A per-city choice that works on hindsight, and what is left of it on 1 January
Choosing a permit model's class encoding from past data had 0.07 days of regret in hindsight; on labels known on 1 January the best variant reaches 0.58.
-
The demolition finding was a censoring artefact: a fold rule fixed in advance helps Chicago, hurts Austin
Seattle's short 2024 demolition build tail is right-censoring at the pull. A class-merge rule fixed in advance helps a borrowed model in Chicago, hurts Austin.
-
Why a work-type label hurt one permit forecast: demolition, not new construction
A work-type label hurt a cross-city forecast of Seattle build times in 2024. The cost sits in demolition permits, and it did not repeat in 2023.
-
Stop naming demolition: one class merge removes two thirds of a borrowed permit model's build-time error
Folding demolition into alteration cuts a transferred permit model's Seattle build-time q0.9 cost from 3.0 to 1.0 days and keeps its issue-time gains.
-
Score build-time tails on labels you can see: a complete-case score reversed our result
On Seattle permits seen in full, a merge that looked like a 2-day build-time q0.9 gain is a 1.7 to 2.5-day loss; IPCW recovers only 10 to 40% of the gap.
-
Austin's 15-day build-time penalty came from 649 Seattle permits
A work-type class helps San Francisco's borrowed build forecast by 14 to 23 days at q0.9 and costs Austin 10 to 17. The cause: which classes the sources fill.
-
A work-type feature helps one city's build-time forecast and costs another's 15 days
On permit cohorts seen in full, a work-type class cuts Seattle's and San Francisco's build-time q0.9 transfer loss by 3 to 16 days and raises Austin's by 15.
-
Does a work-type label help a permit model move between cities? Yes for issue time, not build tails
On Seattle 2024 permits, a work type read from the description cuts a transferred model's upper-tail issue-time error by 3.9 days but adds 2.2 to build time.
-
Do keyword rules read a building permit correctly? 60% before a check, 85% after
Hand labels show keyword rules read Chicago and Seattle permit descriptions right 60% and 78% of the time; a revision reaches 85% and 95% on held-out rows.
-
Why fixing the permit classifier can't make Seattle look more like its source cities
Correcting Seattle permit classes for keyword-rule error leaves the overlap with its source cities at 0.50: 45% of source rows are trade permits.
-
Cross-city permit models share almost no permit classes: what that does to a transfer test
A keyword map leaves Chicago, SF, Austin and Seattle permits in almost disjoint classes, so the cross-city test really tested cost, text length and calendar.
-
Do permit-duration models move between cities? Seattle says it depends on the target
A model trained on Chicago, San Francisco and Austin permits is tested on Seattle's 2024 permits against Seattle's own per-type baseline.
-
Two models each beat a baseline: that still doesn't say which is better
The build-data transfer code now compares class configurations on the same test rows, with a week-block interval on the difference. Four predictions set first.
-
Reading the work type out of a permit description, and four predictions to test it
Chicago and Seattle permit types name a review track, not the work. The loader now reads a work type from the description, with four predictions set first.
-
Seattle permit-duration intervals: a simple lookup table beats the per-type baseline on width
A week-block bootstrap on Seattle's open permit feed shows a three-key lookup table gives narrower 80% intervals than the per-type baseline.
-
Does a model trained on other cities forecast Seattle permit delays? A second test year says only partly
A permit-delay model trained on Chicago, SF and Austin, tested on Seattle in two years. The 2024 gain at every quantile did not repeat in 2023.
-
Three years of cross-city permit forecasts for Seattle, and why permit type cannot travel
A permit-delay model trained on other cities, tested on Seattle in 2022: the prediction held. Chicago and Seattle permit types do not say what work is done.
-
Does a permit model trained on other cities beat Seattle's own history? Half of it did, in one year
Seattle's issue-to-completion lag was the first cell where a model trained on other cities beat the city's own baseline; year two kept only the lower quantiles.
-
A conformal interval that covered 34% instead of 80%, and the one-line cause
On open permit data, adding jitter to a conformalised interval cut its coverage to 34%. A fix to a negative correction restores 89%.
-
Is the simple permit-duration table really narrower? A bootstrap says mostly yes
A week-block bootstrap confirms which 80% permit-duration interval is narrowest in Chicago, San Francisco and Austin, with one near-tie.
-
Does the open-permit duration model hold up in a fourth city, Seattle?
On Seattle's public-domain permit feed a boosted quantile model beats the per-type baseline on pinball loss at three quantiles, for permit and build time.
-
Which permit-duration interval should a tool ship: a table, or a boosted model?
On three cities' open permit feeds, a plain per-type table gives the narrowest calibrated 80% interval in four of five city-target sets.
-
Flattening a boosted permit-duration model into a table throws the gain away
A table of boosted quantile predictions by permit type and cost bucket kept at least half of the model's gain in only one of five city and target sets.
-
Why did jittering the targets make permit-lag intervals narrower than the data?
A conformal fix for tied integer targets collapsed coverage in the same-day-heavy cells; the saved results show where, and a mechanism explains why.