History & human sciences

Can a Catholic's or Protestant's other survey answers predict the beliefs on marriage, sexuality and devotion they were never asked, and do those associations hold for immigrants and across traditions?

research/christian-belief-models

Research programme: in GSS 1972–2024, a respondent's other answers predict an unasked belief or practice item at a median AUC 0.21 above the decade prior (Catholics: weekly Mass 0.86, premarital sex 0.85, same-sex marriage 0.88, abortion 0.75, divorce law 0.69); models trained on third-generation Catholics rank foreign-born Catholics almost as well but miss their levels by up to 11 points

86 tests

What it found

An open research programme on predicting (imputing) religious belief, teaching adherence, devotion and marriage outcomes from survey profiles, and on whether the learned associations transfer between Christian subcultures, decades, immigrant generations and, later, countries.

Milestone 1 uses the GSS 1972–2024 cumulative file (NORC, release 3a; the split-ballot design leaves most items unasked at random). Across 49 tradition × item cells, weighted 5-fold cross-validated models given a respondent’s other answers beat the decade prior by a median 0.21 AUC, with lower log loss in 44 of 49. Among Catholics boosting reaches 0.86 for weekly Mass attendance, 0.85 for “sex before marriage always wrong”, 0.88 for opposing same-sex marriage, 0.75 for abortion and 0.69 for wanting divorce harder to get.

Gradient boosting adds a median +0.003 AUC over logistic regression. A model fitted on third-plus-generation Catholics ranks foreign-born Catholics within 0.03 AUC on 12 of 13 items but misses their level by more than 5 points on 5 (certainty about God by 11), and a Catholic-trained model ranks Evangelical, Mainline and Black Protestants within about 0.01 AUC while under-predicting Evangelical sexual conservatism by up to 17 points: associations travel, levels do not. The GSS has too few Orthodox respondents (27–205 per item) to model.

Milestone 3 ran both principled multiple-imputation baselines on 307 held-out Catholic item-years: chained equations (MICE) miss the year’s share by 4.7 points and the Si–Reiter Bayesian latent-class mixture (DPMPM) by 4.8, both worse than interpolating the neighbouring years (3.4), with Rubin intervals covering 82–83% at a nominal 95% and almost none of the same-sex-marriage years; a drift-aware conformal interval around a neighbour-anchored estimate reaches 95%. Adding the survey decade as one more variable in the mixture cuts its error to 3.55 points (paired −1.26) and same-sex marriage from 20 to 11.5, with Rubin coverage 92%: the missing period, not the imputer, was the barrier, though the period-aware mixture only matches the decade mean and does not beat interpolation.

Show the rest of the summary (7 more paragraphs, 639 words)

Limits: one fold seed, respondent bootstrap rather than design-based variance, shares not yet checked against NORC’s published tables, and global data (EVS/WVS/ISSP) awaits registration. It uses anonymised public survey data only and is not a tool for inferring a named person’s religion.

Milestone 6 (part) splits a transferred model’s error in a group’s share exactly into covariate shift and concept shift over 241 item × group cells: covariate shift, fixable without target answers, is a median 0.8 points; concept shift is 3.5. Borrowing the shift from the items a group was asked does not recover it (−0.35 points, 95% interval −0.74 to +0.01; the pre-registered refutation rule was met), so a rotated item needs a calibration subsample in each group.

How big: with n group members asked the item, the transferred model plus their mean residual (a difference, or prediction-powered, estimator) needs 17% fewer of them than their plain mean (median variance ratio 0.83, 95% interval 0.80–0.85), from 33% for same-sex marriage to 4% for ever divorced; below a break-even n* = var(y − f)/bias2 (median 91) the model alone is better, and a ±3-point share still takes about 700–1,400 asked respondents per group. Given one budget for several subgroups, the split barely matters (Neyman allocation moves the loss under 1%) while the estimator does: shrinking each group’s residual correction with a Fay–Herriot weight whose bias variance is taken from items already fielded halves the group-average squared error at 50 respondents per group, over 73 families of 2–4 subgroups.

For intervals, the robust empirical Bayes interval (Armstrong, Kolesár and Plagborg-Møller 2022) around that shrunk estimate covers the true share 96.8% of the time on average at 50 per group while 17% narrower than the difference estimator’s; the textbook posterior interval averages 93.8% (90.6% for later decades), and no interval protects the 12 of 227 groups biased by more than twice the typical amount (45–64% coverage). Choosing that group’s prior per kind from the other items, without the rotated one, agrees with the by-kind rule in 37 of 39 cases, but the two misses (7% of groups, at 3–5× the error) leave 0.83 of the zero prior’s error, against 0.65 for the in-sample rule fixed by kind in advance.

Fixed in advance and scored on 7 fresh GSS items it was never read from (euthanasia, assisted suicide, death penalty, marijuana, pornography law, teen sex, teen birth control), the by-kind rule keeps about half that gain: 0.85 of the zero prior’s error at 50 per group (bootstrap 0.76–0.92; 0.87 on 6 abortion sub-items), nearly all from 2016–24 groups borrowing the other traditions’ miss (0.38); for traditions and generations the best prior flips between item sets. Under the GSS’s own strata and paired PSUs, design effects on these pooled subgroup shares are small (median 1.06; 1.02 for the model’s residual), but the survey’s own sampling error, which no calibration subsample removes, cuts the difference estimator’s saving on the population share from 17% to 14% at 400 respondents and leaves subsample-only intervals 4–9% too narrow (up to 19% for small groups); an ultimate-cluster variance from the model’s predictions plus the subsample’s answers is within 1–7% of the target.

The same correction carries to the shrinkage sections: the gains are unchanged against the population share (family-median loss 0.471 of the difference estimator’s at 50 per group), but the robust interval from the subsample alone covers 93.0–93.6% at 200–400 respondents, and adding the full sample’s design variance restores 94–97% for 1–3% more width.

Milestone 5: a local open-weights model (Qwen2.5-1.5B-Instruct, CPU) given only a respondent’s background ranks unasked items at AUC 0.57 (transferred boosted model 0.78), misses group shares by a median 33 points zero-shot, and predicts between-group gaps in the right direction (Spearman 0.35) but at about a tenth of their size, so it replaces neither the transferred model nor the calibration subsample.

Research log

Ongoing: each run adds a batch and an entry, newest first. Source: LOG.md.

2026-11-19 · Milestone 6 (part): the pre-registration record recounted by script, milestones 1 to 6

Show 25 earlier entries

2026-11-19 · Milestone 5 (part): does the language model's sense of the gap between groups grow with scale? A 3B point (predictions written before the run; outcome below)

  • Why: the 2026-11-17 entry left one narrow, testable question: a local model knows which way groups differ but predicts a tenth of the gap (slope of predicted on true gap 0.09 for Qwen2.5-1.5B, −0.10 for 0.5B). If the slope grows quickly with scale, the paper's conclusion ("each group needs its own calibration subsample") has an expiry date set by model size; if it grows slowly, the conclusion holds for any model a survey office can run locally. A third point on the same family's scale curve is the cheapest real test. A 7B model in bfloat16 does not fit this session's 15 GB of memory; Qwen2.5-3B-Instruct (6 GB) does.
  • Design: unchanged from 2026-11-17 except the model. Same code (llm.py, score.py), same seed-0 sample of 3,471 respondents, same profiles and questions, same 74 cells; scored label-blind before any outcome is computed. scale.py (new) computes the 3B model's per-cell AUC, level error, predicted gap and LLM-shift estimate exactly as analyse.py does and joins them to the committed cells.csv rows of the other estimators (the 0.5B and 1.5B per-respondent scores are not in git; their per-cell values are). It adds a cell-bootstrap 95% interval (2,000 draws) for each model's slope of predicted on true gap. Licence note: Qwen2.5-3B-Instruct is released under the Qwen Research Licence (research and non-commercial use; the 0.5B and 1.5B are Apache-2.0). It is used here for research only and no weights are redistributed.
  • Predictions: (1) Gap slope of the 3B model between 0.10 and 0.35 (a log-linear extrapolation of the 0.5B and 1.5B points gives about 0.21), with its interval excluding 1; Spearman with the true gap at least 0.30. (2) Zero-shot median AUC between 0.56 and 0.64, and at least 0.06 below GBM-demog (0.699). (3) Zero-shot median absolute level error at least 15 points. (4) 3B-shift beats carrying the source share (median paired abs-error difference below 0) and stays at least 2 points behind GBM-full. (5) By kind, 3B-shift does better than 1.5B-shift on tradition cells (median abs error below 14.78). The milestone's reading is refuted ("a model a survey office can run locally does not carry the concept shift") if the 3B slope is at least 0.5 with a bootstrap interval whose lower end is above 0.3, or if 3B-shift beats GBM-full's median absolute level error.
  • Outcome (experiments/m5_local_llm/results/scale_summary.md, scale_cells.csv, scale_slope.svg; scoring 3,478 respondents took 109 min on 4 CPU cores at 1.9 s each, the analysis 10 s). Pipeline check: all 74 cells match the committed cells.csv (same n<sub>T</sub>, n<sub>S</sub>; largest share difference 8e-17). (1) Partly held: Spearman of the predicted with the true gap rose to 0.556 (p = 3e-7; 1.5B 0.345) and the slope's interval excludes 1, but the slope fell to 0.02 (cell bootstrap 0.01 to 0.04), below the predicted 0.10 to 0.35. (2) Partly held: median AUC 0.5599, a hair under the 0.56 floor (0.560 rounded; interval 0.545–0.577; paired with 1.5B +0.005, −0.017 to +0.026), and 0.111 below GBM-demog (held). (3) Held: zero-shot median level error 61.4 points, almost twice the 1.5B model's 32.8; the model predicts too low in all 74 cells. (4) Held, trivially: 3B-shift beats the source share by a median 0.006 points (0.002 to 0.015) because its predicted gaps are almost zero (median |gap| 0.02 points), and it trails GBM-full by 5.66 (4.33 to 7.97). (5) Held: 3B-shift on tradition cells 14.22 against 1.5B-shift's 14.78. Refutation rule not met (slope 0.02; 3B-shift 12.27 against GBM-full's 5.11). Of 5 predictions, 3 held and 2 held in part.
  • Why the slope fell: Qwen2.5-3B-Instruct answers No to almost everything. 87% of its 40,249 scores are below 0.01 or above 0.99 (nearly all below), and on same-sex relations, adultery and abortion its median P(yes) rounds to 0.0000. Group means on the probability scale then sit near zero, so their difference cannot carry a gap of any size, however well the model orders the groups. The pre-registered metric measured this saturation as much as knowledge.
  • Exploratory, after the outcome above (not pre-registered, so weigh accordingly): on the logit scale (weighted mean logit of P(yes), clipped at 1e-6, the CSV's resolution; 17% of scores sit below 1e-5 and 2% round to 0, so the clip binds for some items) the 3B model's group gap tracks the true logit gap with Spearman 0.597 and slope 1.30 (0.75 to 1.86). That is driven by tradition cells (slope 1.64, Spearman 0.57); generation (0.25), ethnicity (0.14) and period cells (−0.07) carry little. Used as an estimator, expit(logit θ<sub>S</sub> + predicted logit gap) misses by a median 11.57 points, no better than the source share (paired +0.07, −2.78 to +4.25) and 6.41 behind GBM-full (3.09 to 9.49); it helps on certainty about God (10.7 against 23.4) and premarital sex (6.8 against 11.7) and hurts on attendance (20.6 against 5.7) and daily prayer (23.6 against 9.7). So the 3B model does know which traditions differ and by roughly how much in log-odds, but not about generations, Hispanic Catholics or periods, and not well enough to replace a calibration subsample.
  • Check not done: check.py (bfloat16 against float32 on 20 respondents) was killed for memory: float32 weights of a 3B model (12 GB) plus the process did not fit in this session's 15 GB. The 1.5B check agreed at Pearson 0.995; the 3B's scores are unchecked against float32, which matters most for the tiny probabilities the logit analysis uses.
  • Reading: scale does not close the gap in this family at this size. From 1.5B to 3B the model orders groups better but becomes more extreme, so the estimators the paper would use (the probability-scale shift and the difference estimator) gain nothing. The paper's conclusion that each group needs its own calibration subsample holds against three local model sizes. The question for larger models is now two-part: does the logit-scale slope near 1 on tradition pairs persist, and does it appear on generation and period pairs, where the 3B model has none?
  • Next: the same code on a 7B model with a GPU or more memory (score.py would need p written as log-odds to keep the tails); paper Section 4 paragraph and the pre-registration table updated in this run.

2026-11-17 · Milestone 5: can a local open-weights language model stand in for the transferred model or the calibration subsample? (predictions written before the analysis; outcome below)

  • Why: milestone 5 ("a local-LLM imputation baseline, an open-weights model, no paid API, against the tabular models") is the one milestone never started, and the paper's introduction cites LLM opinion imputation (Argyle et al. 2023; Kim and Lee 2023) without a number of its own. The paper's central finding is that a transferred model misses a subgroup's level through concept shift, which the subgroup's other answers cannot recover. A language model is the one predictor here that knows something about the target group without any of its GSS answers: it is told "Religion: Evangelical Protestant" or "Interviewed in 2021" and may know from pre-training how such people answer. So the question is sharper than "is the LLM a good imputer": does it know the shift the tabular model cannot see?
  • Design (experiments/m5_local_llm/): two Apache-2.0 models, Qwen2.5-0.5B-Instruct and Qwen2.5-1.5B-Instruct, on CPU in bfloat16 (llm.py). Each respondent gets a plain-English profile built only from the 15 demographic variables of src/gss.py (DEMOG: year, age, sex, schooling, race, Hispanic origin, generation, region, political views, party, marital status, children, income, raised Catholic, strength of affiliation) plus the tradition; no item answers, as in Argyle et al.'s "silicon samples". Each of 12 items (the 13 of the paper less ever_divorced, whose profile would reveal current marital status) is a yes/no question phrased so that "Yes" is the coded answer; P(yes) is the softmax of the next-token logits of "Yes" and "No". The profile is run once and its key/value cache reused for the 12 questions (checked to agree exactly with an unbatched pass in float32). Sample (sample.py, seed 0): a simple random sample of 300 respondents from each of 12 groups (3,471 respondents in all), scored label-blind by score.py before these predictions were written (no outcome was computed). Eight transfer pairs (generation 3→1, non-Hispanic→Hispanic Catholics, Catholic→Evangelical, Catholic→Black Protestant, Evangelical→Catholic, Mainline→Evangelical, Catholic and Evangelical ≤2014→2016–24); cells with at least 60 sampled target and source respondents asked the item. Compared on the same target rows: the raw LLMs (zero-shot, no GSS answers); the 1.5B model recalibrated on the source sample (weighted logistic on its logit, "Platt"); the milestone-1 boosted model on the 15 demographics only (GBM-demog) and on demographics plus the other items (GBM-full, as in m6), both fitted on the whole source group; the source group's share (SRC); and a level-only estimator, the source share plus the LLM's predicted gap between the groups (LLM-shift = θ<sub>S</sub> + f̄<sub>T</sub> − f̄<sub>S</sub>). Metrics per cell: weighted AUC, level error (points), and var(y − f)/var(y), the fraction of a plain calibration subsample the difference estimator needs (Section 5). Intervals: cell bootstrap, 2,000 draws.
  • Predictions: (1) Ranking: median AUC of the zero-shot 1.5B model between 0.55 and 0.68; GBM-demog at least 0.03 higher and GBM-full at least 0.12 higher (median paired differences); 1.5B above 0.5B by a median of at least 0.02. (2) Level: the zero-shot 1.5B model's median absolute level error is at least twice GBM-full's and at least 8 points. (3) Recalibrated on the source, the 1.5B model's median absolute level error is within 1.5 points of SRC's and no more than 1 point below GBM-full's. (4) As the difference estimator's f: median var(y − f)/var(y) at least 0.95 for the raw 1.5B model and at least 0.92 recalibrated; GBM-demog between 0.85 and 0.95; GBM-full at most 0.88. (5) The shift: across cells the LLM's predicted gap f̄<sub>T</sub> − f̄<sub>S</sub> has Spearman correlation at least 0.3 with the true gap θ<sub>T</sub> − θ<sub>S</sub>, but LLM-shift does not beat GBM-full's median absolute level error by more than 0.5 points overall; on tradition cells LLM-shift beats SRC (smaller median absolute error), on generation and period cells it does not. The milestone's headline reading is refuted (a local model does carry the concept shift the tabular model misses) if LLM-shift beats GBM-full by more than 1 point in median absolute error with a bootstrap interval excluding zero.
  • Outcome (experiments/m5_local_llm/results/summary.md, cells.csv, gap_scatter.svg; scoring 3,478 respondents took 27 min for 0.5B and 66 min for 1.5B on 4 CPU cores, the analysis about 3 min). 74 cells over 8 pairs and 10 items: same-sex marriage and cohabitation never reached 60 sampled respondents asked in both groups, and papal infallibility did in only 2 cells. (1) Held: median AUC of the zero-shot 1.5B model 0.569 (cell bootstrap 0.537–0.589); GBM-demog higher by a median 0.130 (0.110–0.140) and GBM-full by 0.193 (0.162–0.209); 1.5B above 0.5B by 0.051 (0.021–0.058); the 0.5B model is at chance (0.531). (2) Held: the zero-shot 1.5B model misses the group's share by a median 32.8 points (GBM-full 5.1), worst on certainty about God (73 points) and daily prayer (54). (3) Partly held: recalibrated on the source, its median error is 14.05 points against SRC's 12.41, 1.64 apart (predicted within 1.5; paired median +2.17, 0.78 to 2.76), and 7.6 points worse than GBM-full. Recalibration imports the source's level and then does slightly worse than carrying the source share, because the model's ranking is too weak to move it the right way. (4) Held: var(y − f)/var(y) 0.990 raw and 1.000 recalibrated (no calibration saving), GBM-demog 0.927, GBM-full 0.796. (5) Partly held: the 1.5B model's predicted gap between the groups has Spearman 0.345 (p = 0.003) with the true gap, but its slope is 0.09: it predicts a median gap of 2.6 points where the true one is 12.4. LLM-shift beats carrying the source share by a median 0.76 points (paired, 0.12 to 1.83) and loses to GBM-full by 4.70 (3.58 to 6.82). By kind it beats SRC on tradition cells (14.8 against 16.0, as predicted) and also on generation cells (8.3 against 10.9, predicted not to) and ethnicity cells (4.2 against 6.9, no prediction), but not on period cells (7.1 against 6.4). The 0.5B model's predicted gap runs the wrong way (Spearman −0.51). The refutation rule was not met. Of 5 predictions, 3 held and 2 held in part.
  • Reading: a small open-weights model given only a respondent's background is no stand-in for either piece of the paper's design. As an imputer it ranks respondents barely above chance (AUC 0.57, where a boosted model on the same 15 variables reaches 0.70) and its raw probabilities are not calibrated to the survey at all, so as the difference estimator's f it saves nothing. Its one real asset is the one the transferred model lacks: it knows, weakly, which way groups differ (more conservative Evangelicals, less devout recent cohorts). But it knows about a tenth of the size of those gaps, so adding its gap to the source share helps a little over the source share and stays far behind a model fitted to the source group's other answers. The paper's conclusion that each group needs its own calibration subsample stands against this baseline. What it does not settle: these are 0.5B and 1.5B models on CPU. Argyle et al. used a 175B model, and the slope of 0.09 may grow with scale. The question to put to a larger model is narrow and testable with this code: does its predicted gap have a slope near 1?
  • Checks (results/check_Qwen2.5-1.5B-Instruct.txt, first 20 sampled respondents): the cached, batched scoring path matches an unbatched pass of the same tokens in float32 to 1.3e-6. The bfloat16 scores used here agree with float32 at Pearson r 0.995 (median absolute difference 0.009, largest 0.053), and their mean is 1.0 point higher (15.3% against 14.4%). That is small next to the 33-point level error and cannot change any reading above. Per-respondent scores stay out of git, like the GSS microdata (.gitignore); score.py regenerates them.
  • Deviations from the plan: none in the analysis. One limit was fixed before scoring: ever_divorced is left out, because the profile states current marital status.
  • Next: the same test with a 7B to 8B open-weights model (an hour per model on a GPU, or a long CPU run), scored on the gap's slope; then a paragraph in the paper (done in brief in Section 4 of paper/draft.md).

2026-11-17 · Milestone 6 (part): Sections 6 to 8 under the GSS design and against the population share (predictions written before the run; outcome below)

  • Why: the draft's "Before submission" lists "design-based variances for Sections 6 to 8 (Section 5 has one)". Sections 6 to 8 score the shrinkage estimator and its intervals against the target's sample share with Kish variances. The entry of 2026-11-16 found that against the population share the survey's own error V1 (which no calibration subsample removes) cuts D's saving from 17% to 14% at n = 400 and makes subsample-only intervals 4 to 9% too narrow. A shrunk estimate has a much smaller phase-2 variance than D, so the same V1 is a larger share of its error: the shrinkage gain and the robust interval's coverage could erode more than D's did. The GSS host serves the file again (checksums match).
  • Design (experiments/m6_design_shrinkage/run.py): (A, microdata) the 228 cells of m6_allocation (19 pairs, 13 items) refitted exactly as there (check: every bias matches its cells.csv); per cell, ultimate-cluster design variances, with GSS strata and PSUs as in m6_design_variance, of the target's weighted share of y (V<sub>y</sub>), of the transferred model's mean prediction f (V<sub>f</sub>) and of the residual (V<sub>r</sub>), so Cov = (V<sub>y</sub> + V<sub>f</sub> − V<sub>r</sub>)/2. (B, closed form) every group × budget row of m6_prior/results/groups.csv (227 groups, n̄ = 50 to 400, priors P0, COL, ADD, LR and the by-kind RULE). Against the population share, D's error is ε<sub>y</sub> + δ (variance V<sub>y</sub> + v) and the shrunk estimate's is (1 − γ)(b − m) + (1 − γ)ε<sub>f</sub> + γε<sub>y</sub> + γδ, normal with that mean and variance (1 − γ)<sup>2</sup>V<sub>f</sub> + γ<sup>2</sup>V<sub>y</sub> + 2γ(1 − γ)Cov + γ<sup>2</sup>v; the sample-level bias b stands in for the population one. Scored: mean-MSE ratios as in Sections 6 and 8, and exact coverage of the paper's robust EB interval (Kish v) and of a design-aware one that replaces v by v + V<sub>y</sub> in γ, the critical value and the width (the designer's estimate of V<sub>y</sub> was within 1 to 7% in the entry of 2026-11-16; here it is taken as known). Item-cluster bootstrap 95%. Phase 2 stays simple random within the target, as GSS ballots are assigned.
  • Predictions: (1) median V<sub>f</sub>/V<sub>y</sub> over cells between 0.3 and 0.8, and median correlation Cov/√(V<sub>f</sub>V<sub>y</sub>) at least 0.5. (2) FH_prior (P0) mean MSE over D's against the population share at most 0.55 at n̄ = 50 and within 0.03 of the sample-share ratio; at n̄ = 400 between the sample-share ratio and 0.95. (3) The paper's robust EB interval, scored against the population share: mean coverage at least 94% at 50 and between 90% and 94.5% at 400; the 10th percentile below 90% at 400. (4) The design-aware robust interval keeps mean coverage at least 94.5% at every n̄, with median half-width at most 3% above the Kish one at 50 and at most 15% above at 400. (5) RULE over P0 against the population share at 50 within 0.03 of its sample-share 0.648; period COL over P0 at most 0.30 at 50. Refuted (the paper's "halves the loss" does not survive the design) if P0's ratio to D against the population share exceeds 0.60 at n̄ = 50.
  • Outcome (experiments/m6_design_shrinkage/results/summary.md, cells.csv, groups.csv, coverage_by_budget.svg; about 2 min on 4 cores; test_design_shrinkage.py checks the closed form against a 400,000-draw Monte Carlo). Pipeline checks: all 228 cells match m6_allocation (largest bias difference 0.0000 points), and the sample-share MSE recomputed here matches m6_prior's to 3e-13. (1) Refuted: median V<sub>f</sub>/V<sub>y</sub> 0.209 (quartiles 0.123, 0.311), below 0.3; median correlation 0.453, below 0.5. The model's mean prediction varies less across PSUs than predicted. (2) Partly held: P0 over D against the population share is 0.635 at n̄ = 50, within 0.03 of the sample-share 0.641 (held), but not at most 0.55 (refuted), and at 400 it is 0.876, below the sample-share 0.887 rather than between it and 0.95 (refuted: the survey's error helps the shrunk estimate slightly, because it carries only part of V<sub>y</sub>). (3) Partly held: the paper's Kish-variance robust interval covers the population share 96.6% at 50 (held) and 93.6% at 400 (held), but its 10th percentile at 400 is 90.6%, not below 90% (refuted; 89.6% at 200). (4) Partly held: the design-aware interval averages 96.8%, 95.1%, 94.1%, 95.0% (94.1% at 200 misses the 94.5% bar), with median width 1.009 and 1.034 times the Kish one at 50 and 400 (held). (5) Held: RULE over P0 0.649 against 0.648; period COL 0.265. Refutation rule met as written (0.635 > 0.60), but the rule was mis-specified: I set its threshold from the draft's 0.501, which is a median over families of a simulated ratio, and applied it to the ratio pooled over all groups, which is 0.641 under the sample share before any design enters. On the draft's own statistic (family median, computed analytically) the design moves 0.472 to 0.471. The design does not touch the "halves the loss" claim; the mistake was in my prediction, and it is recorded as met rather than rewritten.
  • Reading: the point estimates of Sections 6 to 8 survive the design to within 0.01 in every ratio, because the shrunk estimate inherits (1 − γ) of the error of the model's mean, which is small and only moderately correlated with the sample share's, and γ of the sample share's. At n̄ = 50 phase 1 is 3% of its MSE (4% of D's); at 400, 11% (12%). The intervals are where it matters: built from the subsample alone, the robust EB interval averages 93.0 to 93.6% at 200 to 400 for the population share, 92.5 to 92.6% for generation and period groups at 400 (V<sub>y</sub> a fifth of D's error there). Adding V<sub>y</sub>, the design variance a designer already computes for Section 5, to v in γ, the critical value and the width restores nominal average coverage for 1 to 3% more width. Draft updated: a paragraph closing Section 8, step 3 of the designer's rule, limitations, the pre-registration table (52 predictions: 30 held, 9 partly, 14 refuted) and the abstract; check_numbers.py now ties 100 numbers. Caveats: with-replacement PSU approximation; the sample-level bias stands in for the population one; V<sub>y</sub> taken as known; pooled decades.
  • Citations (same run): the DOI and arXiv hosts answered, and cite_check.py paper/references.bib resolved all 18 ids. One was wrong. Prasad and Rao 1990 (also quoted in the 2026-11-14 entry below) carried doi:10.1080/01621459.1990.10476213, which is Gelfand and Smith's 1990 JASA paper. It is corrected in paper/references.bib to 10.1080/01621459.1990.10475320, which Crossref lists as Prasad and Rao, JASA 85, 163–171. The older log entry is left as written.
  • Next: a second reader recount of the pre-registration table; the journal template, word count and cover letter (owner steps); a second survey for Section 8 (owner registration).

2026-11-17 · Milestone 6 (part): the paper's six figures at print size, each checked against the numbers the text quotes

  • Why: "Figures at print size from the committed CSVs" was the first open item under the draft's "Before submission". The GSS host still refuses this session's proxy, so the design-based variances for Sections 6 to 8 (the other open analysis item) could not be run; the figures need no microdata. No new experiment, so no predictions were written.
  • What (paper/figures.py, paper/figures/fig1..6.{pdf,svg}, test_figures.py, draft Section 12): six figures at Sage column widths (3.5 in single, 7.0 in double; vector PDF with embedded fonts, SVG; Okabe–Ito colours plus distinct markers and line styles for greyscale): (1) covariate- against concept-shift part of each cell's bias; (2) analytic n* against the simulated crossover, log–log with a factor-of-two band; (3) family loss over the difference estimator by budget for six strategies; (4) exact coverage of the three shrinkage intervals against b<sup>2</sup>/τ<sup>2</sup>, with bin means; (5) MSE of each prior over the zero prior's by kind, in-sample against the fresh items; (6) D's error ratio to M against the sample and population shares, with the interval inflation. A rerun writes byte-identical files.
  • Check: before drawing, the script recomputes from the plotted CSV rows 24 numbers the draft quotes from the experiments' summary.md files, and all 24 agree (for example 137 crossover cells, 99% within a factor of two, median ratio 0.71; FH_prior loss 0.501 at n̄ = 50 and 0.885 at 400; robust EB coverage 96.8% at 50 and 63.8% for the 12 groups with b<sup>2</sup>/τ<sup>2</sup> ≥ 4; period COL 0.259 in-sample and 0.383 on fresh items; total-error ratio 0.834 and 0.863). Every cell's  + B̂ equals its bias to 10<sup>−14</sup>. Recomputing Section 5's crossover from sims.csv with the same interpolation rule reproduces m6_calibration's count exactly.
  • What the pictures add to the text: Figure 1 shows the covariate-shift part is not just smaller in the median but near zero for every generation, Hispanic and period cell; only tradition pairs move off the axis. Figure 2 shows the factor-two agreement of n* is systematic, not scattered: almost every cell lies below the equality line, so the formula is a slightly early, not a noisy, rule. Figure 5 puts the instability of the structured priors side by side: for traditions the best prior flips between ADD (13 old items) and LR (fresh items), while the period gain survives on both sets. The figure caption also corrects a naming risk: the "known-bias oracle" is the best fixed weight with a zero prior mean, so an informative prior can beat it (period COL 0.259 against 0.272).
  • Next: design-based variances for Sections 6 to 8 when the GSS host is reachable; the 9 citation ids the proxy blocked; then the journal template (owner step).

2026-11-16 · Milestone 6 (part): the calibration section under the GSS's own design, strata and PSUs (predictions written before the run; outcome below)

  • Why: the paper draft (paper/draft.md, sections 5 and 10, "Before submission") says the calibration numbers ignore the GSS's clustered design: subsamples were simple random draws scored against the target's sample share, with weights-only (Kish) variances. Two consequences are untested. (a) Clustering: the GSS draws paired PSUs within strata (vstrat, vpsu; 2 PSUs per stratum, strata unique to a year), so true variances are larger; the transferred model may remove some of the between-PSU variation, which would make D's saving larger under the design than under Kish. (b) The survey's own error: against the population share, M and D share the phase-1 variance V1 of the full target sample's share, which no calibration subsample reduces, so D's saving on total error must shrink as n grows.
  • Design (experiments/m6_design_variance/run.py): the 241 cells of m6_calibration refitted identically (pipeline check: biases and var(y−f)/var(y) must match its cells.csv). Per cell, on all target rows asked: ultimate-cluster (with-replacement PSU) linearised variance of the weighted share of y and of r = y − f, with every PSU of each stratum counted from the whole file (domain estimation); deff = design / Kish variance. 1972–74 rows carry no design codes and are each their own PSU in a per-year stratum. Per n ∈ {50, 100, 200, 400} (n at most half the rows), 500 simple random subsamples: phase-2 variances V2_M, V2_D around the sample share; total-error ratio (V1 + V2_D)/(V1 + V2_M); V1's share of M's total error; the inflation sqrt((V1 + V2_D)/mean Kish interval variance) by which m6_calibration's D intervals are too narrow for the population share; and a design variance estimate for D a designer could compute (f on every target row, y on the subsample only: ultimate-cluster variance of w(f − f̄)/W<sub>T</sub> + w(r − r̄<sub>s</sub>)/W<sub>s</sub>), scored as its mean over draws against V1 + V2_D. Medians over cells with pair-cluster bootstrap 95%. Synthetic checks in test_design_variance.py (each-row-a-PSU equals Kish; a paired-PSU random-effects simulation is matched within 20%).
  • Predictions: (1) Median deff of y over cells is between 1.1 and 1.8. (2) The residual has the smaller design effect: deff(r) < deff(y) in at least 65% of cells, median deff(r)/deff(y) at most 0.95. (3) The design-based variance ratio V<sub>des</sub>(r)/V<sub>des</sub>(y) has median at most the Kish 0.830. (4) V1's share of M's total error: median at most 0.06 at n = 50 and between 0.08 and 0.35 at n = 400. (5) D's total-error ratio to M: median at most 0.86 at n = 50 and at least 0.86 at n = 400. (6) The Kish D interval is too narrow for the population share by a median factor of at most 1.04 at n = 50 and at least 1.08 at n = 400. (7) The designer's design variance estimate for D has median mean-over-target ratio between 0.85 and 1.25 at every n. Refuted, with the reading that the model does not take out cluster-level variation, if median deff(r)/deff(y) exceeds 1.0; and the paper's section-5 saving overstates the design-based one if the design ratio's median exceeds 0.86.
  • Outcome (experiments/m6_design_variance/results/summary.md, cells.csv, sims.csv, total_ratio_by_n.svg; 241 cells, 931 cell × n rows, about 3 min on 4 cores with OMP_NUM_THREADS=1; a 2-pair --quick run was looked at after these predictions were committed, before the full run). Pipeline check: all 241 cells match m6_calibration (largest bias difference 0.0000 points, variance ratio 0.00000). Design effects are small: median deff(y) 1.058 (pair-cluster bootstrap 1.047–1.071; quartiles 1.004, 1.112), deff(r) 1.024; deff(r)/deff(y) 0.974 (0.966–0.981), below 1 in 78% of cells. The reason is visible in the cells: a subgroup's rows are spread over many PSUs, a median of 2.26 rows per PSU (1,568 PSUs per cell, pooled over decades), so within-PSU correlation has little to act on. Variance ratio var(r)/var(y): Kish 0.829, design 0.811 (0.781–0.834); the largest design gains are on same-sex marriage (0.674 to 0.588) and same-sex relations (0.686 to 0.634). Against the population share, V1's share of M's total error is 0.015, 0.031, 0.059, 0.110 at n = 50, 100, 200, 400 (0.23–0.25 for generation, ethnicity and period groups at 400; 0.05 for traditions); D's total-error ratio to M is 0.834 (0.806–0.855), 0.834, 0.853, 0.863 (0.836–0.882), against 0.83 on phase 2 alone; m6_calibration's D intervals are too narrow for the population share by a median 1.036, 1.039, 1.053, 1.085 (1.17–1.19 for the small groups at 400). The designer-computable design variance for D averages 0.935, 0.966, 0.986, 0.989 of its target (10th–90th percentile over cells 0.87–1.00 at n = 50, 0.92–1.07 at 400); the Kish variance falls to 0.849 at 400. (1) Refuted: deff(y) 1.058 is below the 1.1 floor. (2) Partly held: below 1 in 78% of cells, but the median ratio 0.974 is above 0.95. (3) Held: 0.811. (4) Held: 0.015 and 0.110. (5) Held, narrowly at 400: 0.834 and 0.863. (6) Held, narrowly: 1.036 and 1.085. (7) Held. Neither refutation rule is met.
  • Reading: the paper's section-5 numbers survive the design. For subgroup estimates pooled over many GSS years, clustering adds only about 6% to the variance of a share (2% for the residual), because each PSU holds about two respondents of the group; the model removes a little of that, so the design-based saving is slightly larger than Kish's (19% against 17%). What the simple-random analysis did leave out is the survey's own error: no calibration subsample reduces it, so D's advantage on the population share falls from 17% to 14% as n grows to 400, and Wald intervals built from the subsample alone are 4 to 9% too narrow (up to 19% for small groups). The fix is cheap: the ultimate-cluster variance of the combined linearised values, computable from f on every target row and y on the subsample, is within 1 to 7% of the target. Caveats: with-replacement PSU approximation (no finite-population correction at stage 1); 1972–74 treated as simple random within year; GSS PSUs from one year only cover that year's strata, so pooling decades is what keeps deff small, and a single-year subgroup estimate would see larger effects; subsamples drawn simple random within the target, as GSS ballots are assigned.

2026-11-16 · Milestone 6 (part): paper draft v0, the transfer bound and the designer's rule in one argument

  • Why: milestone 6 is "the transfer bound, and the paper draft". Seven milestone-6 steps (2026-11-12 to 2026-11-16) had landed as separate log entries; none of them said what the paper's argument is, which numbers carry it, or how many pre-registered predictions held. No new experiment was run, so no predictions were written.
  • What (paper/draft.md, paper/references.bib, paper/check_numbers.py): a methods-paper draft for Sociological Methods & Research in ten sections: the exact decomposition b = A + B with the Cauchy–Schwarz bound |A| ≤ √(1 + χ<sup>2</sup>(T‖S)) ‖f − p<sub>S</sub>‖<sub>L2(S)</sub> (the median importance-weight effective sample fraction 0.33 implies a multiplier of about 1.7 in a typical pair, a reading added here, not a new run), calibration size and n*, budget split and shrinkage, robust EB intervals, priors from other cells, the per-kind choice and the fresh-item test, a four-step rule for designers, limitations, and a pre-registration record. check_numbers.py ties 71 quoted numbers to the experiments/m6_*/results/summary.md files (all pass; also run as test_paper_numbers.py).
  • Pre-registration record, counted from each outcome's own verdict in this log: 40 numbered predictions over the seven steps, 23 held, 5 partly held, 12 refuted; one step's refutation rule was met (2026-11-12: the asked items do not carry the unasked item's level shift).
  • Not done: 9 of 17 citation ids could not be looked up from this session (the DOI and arXiv hosts refused the proxy) and are listed in the draft's "Before submission"; figures at print size; a design-based variance; a second survey. Next: a design-based (PSU and strata) variance for the calibration section, which the GSS file carries (vpsu, vstrat), then the figures.

2026-11-16 · Milestone 6 (part): does the by-kind prior rule hold on GSS items it was never read from? (predictions written before the run; outcome below)

  • Why: the entry below found that the kind of group predicts which prior mean the shrinkage estimator should use (period groups: the other traditions' miss on the item, COL; immigrant generations: zero, P0; traditions: the additive pair + item fit, ADD), and that this fixed rule scores 0.65 of the zero prior's mean squared error at 50 calibration respondents per group, but only in-sample: it was read off the same 13 items it was scored on. It closed with "only a second survey can test it honestly". The GSS host, which refused the proxy for the last five entries, served the file again today (checksums match data/SHA256SUMS), so a fresh-item test is possible inside the same survey: the rule is fixed now, then scored on GSS items that were never among the 13.
  • Design (experiments/m6_fresh_items/run.py): fresh binary items, coded to the answer consistent with Catholic teaching. Primary set (7): letdie1 no (euthanasia, CCC 2277), suicide1 no (CCC 2281), cappun oppose (death penalty, CCC 2267 as revised in 2018), grass should not be legal (CCC 2291), pornlaw laws against distribution whatever the age (CCC 2354), teensex always wrong (CCC 2353), pillok disagree that teens 14-16 may get birth control without parental approval (CCC 2370, 2223). Secondary set (6): the abortion sub-items abdefect, abrape, absingle, abpoor, abnomore, abhlth (no; CCC 2271), reported apart because abany, one of the 13 and a model feature, nearly determines them. For each of the 19 pairs (3 Catholic generation, 12 tradition, 4 period) and each fresh item, the transferred model is fitted exactly as in m6_allocation (same gradient boosting, same features: demographics plus the 14 belief items), giving the bias b, N and the residual sd with the Kish design effect. Each fresh item is scored on its own: its column is appended to the existing 19 × 13 bias matrix (m6_allocation/results/cells.csv), and each of its cells gets every prior's m and τ<sup>2</sup> from m6_prior.priors_for with only that cell hidden (the other pairs' misses on the fresh item and all 13 old columns visible, the scenario of the entry below). Groups follow m6_intervals: a cell enters at n̄ when its family has at least two groups and N/2 ≥ n̄. Mean group MSE and robust EB coverage come from the same normal-approximation formulas. Pipeline check first: refitting the 13 old items must reproduce cells.csv biases.
  • Estimators: P0, COL, ADD, LR, the fixed rule RULE (period COL, generation P0, tradition ADD; fixed in the entry below, not re-chosen), the difference estimator D and the known-bias oracle. Item-cluster bootstrap 95% over the 7 primary items.
  • Predictions (primary set): (1) RULE's mean MSE is at most 0.85 of P0's at n̄ = 50 and at most 1.02 at n̄ = 400. (2) Period groups: COL at most 0.60 of P0's at n̄ = 50. (3) Generation groups: P0 is at least as good as each informative prior (COL, ADD and LR each at least 0.95 of P0's at n̄ = 50). (4) Tradition groups: ADD's ratio to P0 at n̄ = 50 has a bootstrap interval that includes 1 (no detectable gain or loss). (5) RULE is at most ADD alone at n̄ = 50. (6) RULE's robust EB mean coverage is at least 94% at every n̄. (7) The secondary (abortion) set agrees in direction: RULE below 1.0 of P0's at n̄ = 50. Refuted, with the reading that the by-kind rule does not generalise beyond the items it was read from, if RULE is not at most 0.95 of P0's at n̄ = 50 on the primary set.
  • Outcome (experiments/m6_fresh_items/results/summary.md, cells.csv, groups.csv, mse_by_budget.svg; 13 fresh items × 19 pairs = 247 cells; under 9 min on 4 cores once OMP_NUM_THREADS=1 was set, after a first launch without it ran 11 min at load 16 and was killed). Pipeline check: refitting the 13 old items reproduces all 228 cells of m6_allocation/results/cells.csv exactly (largest bias difference 0.0000 points). The fresh items' transferred-model misses are of the same order as the old ones (RMS by item from 1.4 to 16.2 points; marijuana's period miss 16.2 and teen sex's 11.3 are the large ones). Primary set, 133 groups: RULE's mean MSE over P0's is 0.846 (item-cluster bootstrap 0.76–0.92), 0.874, 0.901, 0.988 at n̄ = 50, 100, 200, 400; ADD alone 0.899, COL 0.937, LR 0.696 (0.64–0.77). By kind at n̄ = 50: period COL 0.383 (0.21–0.55); generation COL 1.453, ADD 1.289, LR 0.787 (0.61–0.93); tradition ADD 1.059 (0.95–1.18), LR 0.768. Robust EB coverage under RULE 98.5, 96.8, 95.7, 94.9%. Secondary (abortion) set, 114 groups: RULE 0.868 (0.81–0.93) at n̄ = 50, best of the five priors there (LR 0.914). (1) Held, narrowly: 0.846 against the 0.85 ceiling, and 0.988 at 400. (2) Held: 0.383. (3) Refuted: COL and ADD are worse than zero for generations, as predicted, but LR is better (0.787). (4) Held: 1.059, interval includes 1. (5) Held: 0.846 against ADD's 0.899. (6) Held. (7) Held: 0.868. Refutation not met: the fixed rule keeps a 15% gain over the zero prior on items it was never read from (in-sample it was 35%).
  • Diagnostic added after seeing the results (not pre-registered; in the summary so it reproduces): the variant with LR for generations and traditions and COL for period scores 0.655 of P0's on the primary set at n̄ = 50 and 0.858 on the secondary set, but 0.745 in-sample on the 13 old items, where the pre-fixed rule scored 0.648. So which of ADD, LR or zero to use for traditions and generations flips between item sets, and choosing it by error on one set does not carry over to the other. The part that holds on every set is the period rule: on the 2016–24 groups, the other traditions' miss on the same item cuts error to 0.26 (old items), 0.38 (primary) and 0.49 (secondary) of the zero prior's.
  • Reading: the by-kind rule is not an in-sample artefact. Fixed in advance and scored on 13 items it never saw, it keeps roughly half its in-sample gain (0.85 and 0.87 against 0.65) with intervals that stay at nominal coverage. The gain comes almost all from one kind: when a decade's groups all drift together, the other groups' miss is a good guess for the rotated one. For traditions and immigrant generations, no prior beats zero consistently across item sets; LR's win on the fresh items and its two bad folds on the old ones (entry below) are the same instability seen from both sides. For a designer: borrow the other groups' miss for a period shift, and use zero for groups that differ by tradition or generation unless a second item set confirms a structured prior. Caveats: same survey (a fresh-item test, not a second survey); the abortion sub-items nearly follow abany; 7 primary items, so the item bootstrap is coarse; same normal approximation and fitted biases as the entries below.

2026-11-16 · Milestone 6 (part): can the per-kind choice of prior be made without the rotated item, and does it keep the gain? (predictions written before the run; outcome below)

  • Why: the entry below found that the best prior mean for the shrinkage estimator depends on the kind of group (the other traditions' miss on the same item for 2016-24 groups, 0.26 of P0's MSE at n̄ = 50; a zero mean for immigrant generations, where every informative prior hurts; undecided for traditions), but that rule was read off the same 13 items it would be applied to, and the per-cell selector SEL that tried to learn it was too noisy (0.83 of P0's, worse than ADD alone). A designer can only choose with the items already fielded. This run asks whether choosing one prior per kind, from the other 12 items, recovers the by-kind gain honestly, and whether the choice is stable across held-out items.
  • Design (experiments/m6_kindsel/run.py, no microdata; gss.norc.org still refuses the proxy with a 403): leave-one-item-out over the 13 items. For held-out item j, column j is deleted from the 19 × 13 bias matrix (m6_allocation/results/cells.csv); every remaining cell (p′, j′) is predicted by COL, ADD and LR with that cell also hidden, and P0 predicts 0. Per kind, the prior with the smallest mean squared prediction error over that kind's remaining cells is chosen (KSEL). The groups on item j then use the chosen prior's existing m, τ<sup>2</sup>, MSE and coverage from m6_prior/results/groups.csv (those were fitted with only the target cell hidden, as a designer rotating one subgroup's item would have). A stricter variant, KSEL_s, also picks by the same criterion but its evaluation on item j only uses priors that need no other pair's answer on j, so it can pick only P0 (the whole-column scenario). Metrics: mean group MSE over P0's and over D's at n̄ ∈ {50, 100, 200, 400}, by kind, item-cluster bootstrap 95%; how often each kind picks each prior across the 13 folds; robust EB coverage; the same with same-sex marriage left out of the evaluation.
  • Disclosure: I have read every table in m6_prior/results/summary.md, including SEL's per-kind pick shares and the by-kind ratios, before writing these. The predictions below are therefore informed by those in-sample numbers; what is new is whether a choice made without item j agrees with them.
  • Predictions: (1) Period groups pick COL or ADD in at least 12 of 13 folds. (2) Generation groups pick P0 in at least 11 of 13 folds. (3) Tradition groups pick ADD in at least 9 of 13 folds. (4) Over all groups, KSEL's mean MSE at n̄ = 50 is at most 0.70 of P0's, at most ADD alone's 0.725, and no more than 1.03 of P0's at n̄ = 400. (5) Leaving same-sex marriage out of the evaluation (its 13 folds' choices unchanged), KSEL is still at most 0.95 of P0's at n̄ = 50. (6) KSEL's robust EB mean coverage is at least 94% at every n̄. Refuted, with the reading that the by-kind rule was an in-sample artefact, if KSEL is not at least 5% below P0 in mean MSE at n̄ = 50.
  • Outcome (experiments/m6_kindsel/results/summary.md, choices.csv, groups.csv, mse_by_budget.svg; 227 groups, 13 folds, 39 s on 4 cores). Choices across the 13 folds: period COL 13; generation P0 12, LR 1 (held-out ever divorced); tradition ADD 12, LR 1 (held-out same-sex marriage). Mean MSE over P0's at n̄ = 50, 100, 200, 400: KSEL 0.826 (item-cluster bootstrap 0.74–0.95), 0.866, 0.917, 1.045; ADD alone 0.725, 0.768, 0.831, 1.017. By kind at n̄ = 50: period 0.259, generation 1.150, tradition 1.247. Without same-sex marriage in the evaluation: KSEL 0.863 at n̄ = 50 (ADD 0.944, COL 0.977, LR 0.818). Robust EB mean coverage 97.3, 95.2, 93.9, 94.3%. (1) Held: period picks COL in 13 of 13. (2) Held: generation picks P0 in 12 of 13. (3) Held: tradition picks ADD in 12 of 13. (4) Refuted: KSEL is 0.826 at n̄ = 50, not at most 0.70, and worse than ADD alone; it is 1.045 at n̄ = 400, above the 1.03 allowed. (5) Held: 0.863 without same-sex marriage. (6) Narrowly refuted: 93.9% at n̄ = 200. The refutation rule is not met: KSEL is 17% below P0 at n̄ = 50, so the per-kind rule is not an in-sample artefact, but most of the in-sample gain is lost.
  • A correction to the entry below: SEL is not a per-cell choice in practice. Every prior's τ<sup>2</sup> is a mean over the same kind's cells on the other items, so it barely depends on which cell of the column is the target, and SEL makes the same pick for every group of a kind on an item. KSEL and SEL agree on all 227 groups here. So SEL's 0.83 is not noise from choosing cell by cell. It comes from two consistent choices on two items, and the entry below's reading ("too noisy to choose among priors cell by cell") was wrong.
  • Diagnostic added after seeing the results (not pre-registered; in the summary so it reproduces): the in-sample rule (period COL, generation P0, tradition ADD) scores 0.648, 0.709, 0.792, 0.993 of P0's. KSEL departs from it on 15 of 227 groups, the 12 tradition groups on same-sex marriage and 3 generation groups on ever divorced. Those 15 groups (7% of the total) add +0.161 of all groups' P0 MSE under KSEL. Under the rule they add −0.017. Their MSE is 4.6 and 3.1 times P0's under the LR prior KSEL picked for them. In both folds LR wins on the other 12 items by a small margin (tradition RMS 4.71 against ADD's 6.42 and P0's 7.00; generation 4.85 against P0's 5.12), then misses badly on the held-out item.
  • Reading: the kind of group predicts which prior to use, and that choice can be made honestly without the rotated item: 37 of 39 kind-folds agree with the in-sample rule. What fails is the selection criterion. Mean leave-out squared error over 12 items is dominated by one or two large cells, and dropping the dominant item (same-sex marriage for traditions) moves the winner to the rank-2 fit, which then fails on exactly that item. For a designer, the safe version is a rule fixed by kind ahead of time, not one re-chosen by error on the items already fielded. On this data that rule gets about 0.65 of the zero prior's error at 50 respondents per group, against 0.83 for re-choosing. But that 0.65 is in-sample, and only a second survey can test it honestly. Caveats: 13 items and 19 pairs, so one item can decide a choice; same normal approximation and fitted biases as the entries below; no microdata (the GSS host still refused the proxy).

2026-11-15 · Milestone 6 (part): can the other subgroups' and items' misses tell the shrinkage estimator which group is biased? (predictions written before the run; outcome below)

  • Why: the two entries below found that shrinking each group's correction toward the transferred model (FH_prior, zero prior mean, τ<sup>2</sup> from the same kind's biases on other items) halves the group-average MSE at 50 respondents per group, but the oracle that knows each group's bias reaches 0.26 of the difference estimator's loss against FH_prior's 0.50, and the robust EB interval covers the 12 groups with b<sup>2</sup> > 4τ<sup>2</sup> only 64% of the time. Both gaps are about knowing which group is biased. The bias matrix (19 source→target pairs × 13 items, m6_allocation/results/cells.csv) may carry that information: a 2016-24 drift on an item is likely shared across traditions, and a target tradition's level gap likely repeats across sources. Fay and Herriot (1979, doi:10.1080/01621459.1979.10482505) shrink toward a regression prediction rather than zero; this run asks which predictor, built only from biases the designer would already know, earns its place.
  • Scenario: item j is rotated off one subgroup's ballot (target cell p, j), while the other subgroups and the other items stay asked in full, so every other cell's bias is known. A stricter variant hides the whole column j (all subgroups lose the item at once), which leaves only the pair's misses on other items.
  • Design (experiments/m6_prior/run.py, no microdata; same normal approximation, per-group sd and budgets as m6_intervals/results/groups.csv): each prior gives a mean m<sub>pj</sub> and a variance τ<sup>2</sup>; the composite is E0 + m + γ(D − E0 − m), whose error given b is (1 − γ)(b − m) + γ√v Z, so MSE and the coverage of the naive and robust EB intervals (Armstrong, Kolesár and Plagborg-Møller 2022, doi:10.3982/ECTA18597) are exact as before with b replaced by b − m. τ<sup>2</sup> for a prior is the mean squared error of that same prior on the same kind's cells for the other items, each of those predictions also made without its own cell (nested leave-out, so no cell's own bias enters its prior). Priors: P0 (m = 0, the current FH_prior); GRP (m = 0, τ<sup>2</sup><sub>p</sub> = the pair's own mean b<sup>2</sup> on other items, a per-group scale); COL (m = mean bias of the same kind's other pairs on item j); ADD (two-way additive pair + item effects within the kind, fitted by alternating means on the known cells); LR (rank-2 hard-impute of the column-centred 19 × 13 matrix). Strict variant: P0 and GRP only (COL, ADD and LR need column j). Metrics per n̄ ∈ {50, 100, 200, 400}: mean group MSE over D's and over P0's, by kind; mean robust-interval coverage, share below 80%, and coverage of the 12 large-bias groups as defined under P0; item-cluster bootstrap intervals for the MSE ratio; oracle (m = b) as the floor.
  • Disclosure: before writing these predictions I looked at two correlations in cells.csv: a cell's bias against the mean of its pair's biases on the other items (−0.38 generation, −0.35 period, +0.53 tradition) and |b| against the pair's mean |b| elsewhere (−0.05). Nothing else was computed.
  • Predictions: (1) For period groups the best of COL/ADD/LR cuts the mean MSE at n̄ = 50 to at most 0.80 of P0's: the 2016-24 drift is shared across traditions. (2) Over all groups, the best informative prior's mean MSE at n̄ = 50 is at most 0.85 of P0's and its advantage shrinks with n̄ (ratio closer to 1 at 400 than at 50). (3) GRP is within 5% of P0's mean MSE at every n̄ (the scale of a pair's misses barely repeats across items). (4) The robust EB interval around the best prior keeps mean coverage at least 94% at every n̄, and covers the 12 large-bias groups at least 75% of the time at n̄ = 50 (P0: 64%). (5) LR is no better than ADD by more than 0.03 in the overall ratio: the matrix is too small for a rank-2 fit to beat an additive one. Refuted, with the reading that other groups' and items' misses carry no usable information about a subgroup's bias, if no informative prior beats P0 by at least 5% in mean MSE at n̄ = 50.
  • Outcome (experiments/m6_prior/results/summary.md, groups.csv, mse_by_budget.svg; 227 groups, 835 group-budget rows, about 4 minutes on 4 cores). Ratios are mean group MSE over P0's (item-cluster bootstrap 95%) at n̄ = 50, 100, 200, 400: COL 0.77 (0.52–1.03), 0.79, 0.84, 1.01; ADD 0.73 (0.46–1.05), 0.77, 0.83, 1.02; LR 0.82 (0.74–0.89), 0.87, 0.92, 1.00; GRP 1.05, 1.05, 1.05, 1.00; the known-bias oracle 0.53, 0.55, 0.60, 0.74. Over D's at n̄ = 50: P0 0.64, ADD 0.47, oracle 0.34 (a mean over groups; the earlier entry's 0.50 was a median over families). By kind at n̄ = 50: period COL 0.26 (0.10–0.74), ADD 0.30, against an oracle of 0.27; tradition ADD 0.91 (0.74–1.11), COL 1.05; generation COL 1.53 (1.44–1.67), ADD 1.63, LR 1.02. (1) Held: for 2016-24 groups the other traditions' miss on the same item cuts the MSE to 0.26 of P0's, essentially the oracle's 0.27; the prior's RMS error is 4.7 points against the 11.7-point bias it replaces. (2) Held on the point estimate, not cleanly: ADD 0.725 at n̄ = 50, shrinking toward 1 with n̄ (1.02 at 400), but the item-cluster interval reaches 1.05, and the gain comes from the period groups; on generation groups every informative prior except LR makes things worse by half. (3) Narrowly refuted: GRP is worse than P0 by 5.0, 5.4, 4.5 and 0.3% (outside the 5% band at n̄ = 100): a pair's own bias scale on other items does not help and slightly hurts. (4) Held: robust EB coverage around ADD is 98.4, 96.5, 95.4 and 94.8%, and the 12 large-bias groups are covered 93.3% of the time (P0 63.8%), at a half-width no wider than P0's (11.3 against 11.4 points at n̄ = 50); their MSE falls to 0.27 of P0's. (5) Refuted in the other direction: LR (0.82) is worse than ADD (0.73), not merely no better. A unit test found why rank-2 completion of the column-centred matrix is weak here: centring adds a rank, so a single hidden cell is not identified by an exact fit (test_prior.py). The refutation rule was not met.
  • Diagnostic added after seeing the results (not pre-registered): SEL picks, per target, the prior with the smallest nested leave-out error on the same kind's other items. It picks COL for every period group, P0 for 92% of generation groups and ADD for 92% of tradition groups, yet its overall ratio is 0.83, worse than ADD alone, because the 8% of tradition cells where it picks LR lose 4.6× P0's MSE. Leave-out error over 12 other items is too noisy to choose among priors cell by cell.
  • Reading: the shared 2016-24 drift is the information that tells the estimator which group is biased. When an item is rotated off one tradition's recent ballots, the other traditions' recent miss on that item predicts this tradition's miss to within 4.7 points RMS, and shrinking toward it reaches the oracle's error at 50 respondents and keeps 23% of P0's error saved at 400. For immigrant generations the other generations' misses point the wrong way (the −0.38 correlation seen before the run), and for traditions the pair-plus-item structure helps only within noise. So the practical rule is by kind, chosen ahead of time and now supported on this data: borrow the other groups' miss for period groups, keep a zero prior for generations, and treat traditions as undecided. Two checks made after the results (not pre-registered) bound this. One item dominates: same-sex marriage, whose 2016-24 groups are missed by 35 to 43 points, which P0's robust interval never covers (0%) and ADD's covers 99.5 to 100%. Dropping that item, COL still brings the period groups to 0.54 of P0's MSE at n̄ = 50 (0.33–0.82; oracle 0.48), but the all-groups gain vanishes (COL 0.98, ADD 0.94, LR 0.82). And the 12 large-bias groups are 7 tradition, 3 period (all same-sex marriage) and 2 generation cells: ADD lifts 5 of the 7 tradition cells' robust coverage (to 0.89 to 0.995) and lowers both generation cells' (to 0.78 and 0.90), because their god-certainty misses run opposite to the other generation's. Caveats: the scenario assumes every other cell's bias is known exactly (other groups asked in full), so it is an upper bound for a design that rotates an item off several groups at once; no microdata (the GSS host answered this session but the run reused the stored biases); one boosted model and one seed; a by-kind rule chosen from 45 period cells in 4 traditions needs checking on another survey or a later GSS wave. Next: the scenario with the whole family's column hidden, where only the pairs' other items and other kinds' cells remain; then the transfer-bound section of the paper.

2026-11-14 · Milestone 6 (part): can the shrinkage estimator's interval be trusted in each subgroup? (predictions written before the run; outcome below)

  • Why: the entry below found that shrinking each group's residual correction toward the transferred model (FH_prior, τ<sup>2</sup> from the same kind of group's biases on other items) halves the group-average MSE at 50 respondents per group, and flagged that its naive interval was not checked. A designer who reports a subgroup share needs an interval, and a shrunk estimate is biased by (1 − γ)b in a group whose transferred-model bias is b, so the textbook interval ±1.96√(γv) (the posterior sd when τ<sup>2</sup> is known; Prasad and Rao 1990, doi:10.1080/01621459.1990.10476213, add terms only for an estimated τ<sup>2</sup>) can cover far below 95% in exactly the groups that matter. The robust empirical Bayes interval of Armstrong, Kolesár and Plagborg-Møller (2022, doi:10.3982/ECTA18597) widens the critical value so that coverage averaged over groups is at least 95% for any bias distribution with the assumed second moment, which here comes from other items rather than the groups themselves.
  • Design (experiments/m6_intervals/run.py): no microdata (gss.norc.org refused the proxy this session); it reuses the per-group bias b, residual sd σ (with the Kish design effect) and budgets of m6_allocation/results/groups.csv and the per-cell biases of cells.csv for τ<sup>2</sup> exactly as that run set it. Under the normal approximation the FH_prior error in group g is (1 − γ)b + γ√v Z, v = σ<sup>2</sup>/n, so each interval's coverage given b is computed exactly rather than simulated. Intervals: D (difference estimator ± 1.96√v, the 95% reference), FH_naive (± 1.96√(γv)), FH_robust (AKP critical value for normalised-bias second moment v/τ<sup>2</sup>, times γ√v) and FH_cmse (± 1.96 times the root of the plug-in conditional MSE γ<sup>2</sup>v + (1 − γ)<sup>2</sup>max(0, r̄<sup>2</sup> − v), integrated over r̄). Per n̄ ∈ {50, 100, 200, 400}: mean coverage over groups, share of groups below 80%, coverage by kind, and median half-width over D's. Check of the approximation first: its RMSE against the simulated FH_prior RMSE of the same groups.
  • Predictions: (1) the analytic RMSE is within 10% of the simulated one in at least 90% of group-budget rows. (2) FH_naive's mean coverage is 85 to 92% at n̄ = 50, with at least a quarter of groups below 80%. (3) FH_robust's mean coverage is at least 93% at every n̄, with at most 10% of groups below 80% at n̄ = 50, and its median half-width is 0.70 to 0.95 of D's at n̄ = 50. (4) FH_cmse covers within 2 points of 95% on average but its median width is above 0.95 of D's, giving back the gain. (5) Period groups, whose biases are largest, have the lowest FH_naive coverage of the three kinds. Refuted, with the reading that an outside τ<sup>2</sup> cannot support intervals, if FH_robust's mean coverage is below 90% at n̄ = 50.
  • Outcome (experiments/m6_intervals/results/summary.md, groups.csv, coverage_by_budget.svg; 227 groups in 73 families, 835 group-budget rows, 7 s on one core). Mean coverage over groups at n̄ = 50, 100, 200, 400: FH_naive 93.8, 93.7, 93.7, 94.9%; FH_robust 96.8, 95.1, 94.1, 95.0%; FH_cmse 89.1, 92.1, 93.9, 94.5% (D 95% by construction). Median half-width over D's at n̄ = 50: FH_naive 0.71, FH_robust 0.83, FH_cmse 0.70 (D 13.6 points, FH_robust 11.4). (1) Narrowly refuted: 87.2% of rows are within 10% of the simulated RMSE (93% at n̄ = 50, 77% at 400). The miss grows with n because the simulation drew without replacement from the N asked rows; with the factor 1 − n/N (a diagnostic added after seeing this, not pre-registered) 94.7% are within 10%. (2) Refuted: the naive interval undercovers far less than predicted, 93.8% on average at n̄ = 50 with 6% of groups below 80% (predicted 85 to 92% and a quarter). (3) Held: FH_robust covers at least 94.1% on average at every budget, 3.1% of groups below 80% at n̄ = 50, half-width 0.83 of D's. (4) Refuted in the other direction: the plug-in conditional-MSE interval is the worst on coverage (89.1% at n̄ = 50, 23% of groups below 80%) and no wider than naive (0.70 of D's), since its floor γ<sup>2</sup>v sits below the posterior variance γv whenever r̄<sup>2</sup> < v. (5) Held: period groups have the lowest naive coverage (90.6% against 94.1% for generation and 94.8% for tradition). The refutation rule was not met.
  • Reading: the average-coverage problem is small because τ<sup>2</sup> taken from other items of the same kind matches the groups' own mean squared bias (RMS bias 5.2, 11.7 and 7.0 points against √τ<sup>2</sup> 5.2, 11.7 and 7.0 for generation, period and tradition), and that match is what the empirical Bayes argument needs. The per-group problem is concentrated: the 12 groups whose bias exceeds 2τ (b<sup>2</sup>/τ<sup>2</sup> > 4) are covered 45% of the time by the naive interval and 64% by the robust one, while groups with b<sup>2</sup> ≤ τ<sup>2</sup> are over-covered (98 to 99%). So for a designer, the robust EB interval is the one to report: it keeps average coverage at 95% for about 0.83 of the difference estimator's width at 50 respondents per group, and it is the only one of the three whose lowest decile stays above 92% at every budget. What it cannot do is protect a group the transferred model misses by more than twice the typical bias; a per-group bias prior (covariates, drift on other items) or a flag on |r̄| is the next lever. Caveats: coverage is exact only under the normal approximation with v known (the simulation's γ used an estimated v); τ<sup>2</sup> treated as known, so no Prasad-Rao term; no microdata this session, so no draw-level coverage check on the GSS ballots themselves; one boosted model and one seed, carried over from the allocation run.

2026-11-13 · Milestone 6 (part): how should a fixed calibration budget be split across subgroups, and does shrinking toward the transferred model beat the difference estimator? (predictions written before the run; outcome below)

  • Why: the entry below sized a calibration subsample for one group at a time and found the difference estimator D saves about a sixth of it, with a break-even n* = σ<sub>Y−f</sub><sup>2</sup>/b<sup>2</sup> (median 91) below which the transferred model alone (E0, bias b) is better. A designer rotating item j off several subgroups' ballots has one budget B for all of them. Two levers remain: where to put the respondents (allocation) and how far to trust E0 in each group (a composite E0 + γ<sub>g</sub>(D − E0), the small-area estimator of Fay and Herriot 1979, doi:10.1080/01621459.1979.10482505, with γ<sub>g</sub> = τ<sup>2</sup>/(τ<sup>2</sup> + v<sub>g</sub>)). With known b<sub>g</sub> and residual sd σ<sub>g</sub> the composite's MSE is b<sup>2</sup>v/(b<sup>2</sup> + v), v = σ<sup>2</sup>/n, and minimising its sum over groups under Σn<sub>g</sub> = B gives a water-filling rule n<sub>g</sub> = max(0, σ<sub>g</sub>/√μ − n*<sub>g</sub>): Neyman allocation (Neyman 1934, doi:10.2307/2342192) minus each group's break-even, so groups the model already fits get nothing.
  • Design (experiments/m6_allocation/run.py): the milestone-1 boosted model and the same source→target cells as the entry below, grouped into families that share an item and a kind: tradition (one source, its 3 other traditions as targets), period (the 4 traditions' 2016-24 targets) and generation (Catholic gen1, gen2 and gen3 targets); the single Hispanic cell has no family. Budget B = G × n̄ with n̄ ∈ {50, 100, 200, 400}, no group above half its asked rows. R = 300 draws of simple random subsamples per family and n̄. Loss = mean over the family's groups of MSE against each group's full weighted share. Strategies: M_eq (plain subsample mean, equal split); D_eq (difference estimator, equal split; the reference); D_ney (D, Neyman split on the oracle σ<sub>g</sub>); C_or (oracle composite with known b and σ, water-filling split; a lower bound, not feasible); FH_fam (equal split, composite with τ<sup>2</sup> estimated by moments across the family's groups from the subsamples); FH_prior (equal split, τ<sup>2</sup> = mean b<sup>2</sup> of the same kind's cells on the other items, as a designer would take from items already fielded); FH2 (half the budget as an equal pilot, the rest water-filled on the pilot's posterior b<sup>2</sup> and σ, then FH_prior). Medians over families, item-cluster bootstrap intervals.
  • Predictions: (1) D_ney's loss is within 3% of D_eq's (median ratio 0.97 to 1.00) at every n̄: residual sds of binary items vary too little for Neyman to matter. (2) The oracle C_or's median loss ratio to D_eq is at most 0.60 at n̄ = 50 and between 0.75 and 0.95 at n̄ = 400. (3) FH_prior captures at least half the oracle's gain at n̄ = 50 and 100 (ratio ≤ (1 + C_or)/2) and is no worse than D_eq by more than 2% (median) at n̄ = 400. (4) FH_fam is worse than FH_prior at n̄ = 50 by at least 0.05 in median ratio, because 3 or 4 groups estimate τ<sup>2</sup> badly. (5) FH2's loss is within 5% of FH_prior's at every n̄: allocation adds little beyond shrinkage. (6) Shrinkage trades the worst group for the average: at n̄ = 50 the median ratio of FH_prior's worst-group RMSE to D_eq's is above 1. Refuted, with the reading that shrinking toward a transferred model does not help small subgroups, if FH_prior's median loss ratio is 0.97 or more at n̄ = 50.
  • Outcome (experiments/m6_allocation/results/summary.md, families.csv, groups.csv, loss_by_budget.svg; 73 families of 2 to 4 groups, 13 items, 300 draws per family and budget, under a minute on 4 cores with OMP_NUM_THREADS=1). Median loss ratio to D_eq at n̄ = 50, 100, 200, 400: D_ney 0.993, 0.997, 0.996, 0.994; oracle C_or 0.26, 0.35, 0.46, 0.56; FH_prior 0.50 (item-cluster bootstrap 95% 0.44 to 0.55), 0.69, 0.80, 0.89; FH2 0.52, 0.69, 0.83, 0.91; FH_fam 0.66, 0.92, 1.03, 1.04; the plain mean M_eq 1.23 throughout. In points (root mean group MSE, median over families) at n̄ = 50: D_eq 6.75, FH_prior 4.66, C_or 3.48; D_eq needs about 100 per group to reach FH_prior's error at 50. (1) Held: Neyman allocation changes the loss by under 1%. (2) Half held: the oracle cuts the loss to 0.26 at n̄ = 50 (predicted at most 0.60), but at 400 it is still 0.56, far below the 0.75 to 0.95 predicted; knowing each group's bias stays worth a lot even with large subsamples, because some groups' biases are small enough that water-filling gives them nothing (40% of groups at n̄ = 50, 18% at 400) and others' are large enough to dominate. (3) Mostly held: FH_prior 0.50 at n̄ = 50 is inside the half-of-oracle line (0.63), misses it narrowly at 100 (0.685 against 0.675), and is 0.89 at 400, better than D_eq rather than the "no worse than 2%" predicted. (4) Held: estimating τ<sup>2</sup> from the family's own 3 or 4 groups costs 0.16 at n̄ = 50 and is worse than D_eq from n̄ = 200 (1.03 to 1.04), since the moment estimator often hits 0 and shrinks a biased group all the way to E0. (5) Held: FH2's pilot-then-water-fill loss is 2 to 3% above FH_prior's at every budget (mean ratio 1.02 to 1.03): with estimated biases, allocation adds nothing and costs a little. (6) Refuted: shrinkage does not trade the worst group for the average; FH_prior's worst-group RMSE is 0.75 of D_eq's at n̄ = 50 and 0.95 at 400. By kind at n̄ = 50, FH_prior's ratio is 0.35 for generation families, 0.47 for tradition and 0.61 for period, where the 2016-24 biases (RMS 11.6 points) are the largest. The refutation rule was not met.
  • Reading: for a designer rotating an item off several subgroups' ballots, the lever is the estimator, not the allocation. Equal splits are fine: Neyman allocation on the residual sd moves the loss by under 1%, and water-filling on pilot estimates of the bias makes it slightly worse. Shrinking each group's residual correction toward zero by γ = τ<sup>2</sup>/(τ<sup>2</sup> + v), with τ<sup>2</sup> set from the same kind of group's biases on items already fielded in full, halves the group-average MSE at 50 respondents per group, which is roughly the difference estimator's error with twice the respondents, and still saves 11% at 400. τ<sup>2</sup> has to come from outside the family: three or four groups cannot estimate it, and the family-level version ends up worse than no shrinkage at moderate budgets. The large gap to the oracle (0.26 against 0.50 at n̄ = 50) is the value of knowing which groups are biased; a better per-group prior for b (from the shift decomposition's covariate part, or the group's drift on other items) is where the rest of the gain sits. Caveats: point error only; Fay-Herriot intervals and their coverage were not assessed, and a shrunk estimate's naive interval is expected to undercover in groups with large bias (not tested here); simple random subsamples within groups rather than the GSS's clustered ballots; one boosted model and one seed; the full asked target stands in for the group's population; families have 2 to 4 groups, so the gains are about small sets of subgroups, not a many-area small-area problem. Next: Fay-Herriot intervals with an honest MSE estimate (Prasad and Rao 1990) and their coverage per group, a per-group prior from covariates, and the paper's section on the bound.

2026-11-12 · Milestone 6 (part): how big a calibration subsample does a rotated item need in each group, and does the transferred model shrink it? (predictions written before the run; outcome below)

  • Why: the entry below found that a transferred model's level error is mostly concept shift (median 3.45 points) that the group's other answers cannot fix, so a rotated item needs a calibration subsample in each group that is asked it. The number a survey designer needs is how big. With a subsample of n target respondents asked item j, two design-consistent estimators of the target's share θ<sub>j</sub> exist: M, the subsample's own weighted mean (variance σ<sub>Y</sub><sup>2</sup>/n), and D, the difference (model-assisted, prediction-powered) estimator E<sub>T</sub>[f<sub>j</sub>] over all target rows plus the subsample's mean residual Y − f<sub>j</sub> (Särndal, Swensson and Wretman 1992; Angelopoulos et al. 2023, doi:10.1126/science.adi6000), variance σ<sub>Y−f</sub><sup>2</sup>/n. So D needs a fraction σ<sub>Y−f</sub><sup>2</sup>/σ<sub>Y</sub><sup>2</sup> of M's sample for the same precision, and it beats the label-free transfer E0 (bias b, no variance) once n > n* = σ<sub>Y−f</sub><sup>2</sup>/b<sup>2</sup>. experiments/m6_calibration/run.py refits the milestone-1 boosted model on the source group for the same 20 pairs and items as the entry below, then draws R = 500 simple random subsamples of n ∈ {25, 50, 100, 200, 400} from the target rows asked j (n at most half of them), and scores M, D, PPI++ (D with a power-tuning factor λ̂ = cov(Y, f)/((1 + n/N) var(f)) estimated on the subsample; Angelopoulos, Duchi and Zrnic 2023, arXiv:2311.01453) and R (f recalibrated by a logit intercept fitted on the subsample, then averaged over all target rows) against the target's full weighted share, with Wald 95% intervals (Kish-linearised weighted variance, finite-population correction) for M and D. A test run on synthetic data checked the code; these predictions were written before any GSS output was read.
  • Predictions: (1) D has lower RMSE than M in at least 85% of cell × n combinations, and the median variance ratio σ<sub>Y−f</sub><sup>2</sup>/σ<sub>Y</sub><sup>2</sup> over cells is between 0.65 and 0.85 (D saves 15 to 35% of the calibration sample); (2) savings are largest for weekly attendance and the sexual-morality cluster (median saving at least 30% over their cells for premarital sex, same-sex relations, same-sex marriage and weekly attendance) and smallest for ever divorced and divorce law (under 10%); (3) the median break-even n* where D overtakes E0 lies between 50 and 300, and in at least 80% of cells whose simulated crossover falls inside the grid the analytic n* is within a factor of 2 of it; (4) PPI++ and R are within 5% of D's RMSE (median ratio over cells) at every n; (5) D's Wald 95% interval covers θ in 93 to 96% of draws pooled over cells at n ≥ 100, and below 93% at n = 25. Refuted, with the reading that a transferred model does not shrink the calibration sample a rotated item needs, if the median variance ratio is 0.95 or more.
  • Outcome (experiments/m6_calibration/results/summary.md, cells.csv, sims.csv, rmse_by_n.svg; 241 cells over 20 pairs, 1,172 cell × n combinations, 500 subsamples each, about 4 minutes on 4 cores). Median variance ratio σ<sub>Y−f</sub><sup>2</sup>/σ<sub>Y</sub><sup>2</sup> 0.830 (pair-cluster bootstrap 95% 0.798 to 0.852; quartiles 0.73 to 0.90; below 1 in 98% of cells), so the difference estimator needs 17% fewer calibration respondents than the plain subsample mean for the same precision. Median RMSE in points at n = 25, 50, 100, 200, 400: plain mean 11.0, 7.9, 5.4, 3.7, 2.5; difference 9.7, 6.9, 4.8, 3.3, 2.3; the transfer alone (|bias|) 4.1 at every n. (1) Held: D's RMSE is below M's in 96% of combinations and the median ratio 0.830 is inside 0.65 to 0.85. (2) Half held: the smallest savings are ever divorced (4%) and divorce law (7%), under 10% as predicted (afterlife and papal infallibility are as low, 6%, which was not predicted); the four high-AUC items save 28% pooled, short of the 30% predicted (same-sex marriage 33%, same-sex relations 31%, weekly attendance 25%, premarital sex 22%). (3) Held: median analytic n* 91 (quartiles 27 to 317; tradition pairs 58, generation 153, period 159, Hispanic 204); in the 137 cells whose simulated crossover lies inside the grid, the analytic n* is within a factor 2 of it in 99% (median ratio 0.71, the formula leaning early because it ignores the finite-population correction and weight variation). (4) Held: PPI++ and logit recalibration are within 0.3% of D's RMSE at every n (median ratios 0.997 to 1.002); with a model that is well-scaled in ranking, power tuning chooses λ near 1 and there is nothing to gain. (5) Held: D's Wald 95% interval covers 93.9% of draws pooled at n ≥ 100 (93.3, 94.1, 94.5% at 100, 200, 400) and 90.0% at n = 25; the worst single cell is 77% at n = 25 and 87% at 100. The refutation rule was not met.
  • Reading: a transferred model does shrink the calibration sample a rotated item needs, but by about a sixth, not by half. The ceiling is the item's predictability: even at AUC 0.85 to 0.88 a binary answer keeps most of its variance (the residual variance ratio is about 1 − R<sup>2</sup> of the probability model), so the best items save about a third and the hard ones (divorce law, ever divorced, afterlife) almost nothing. The more useful number for a designer is the break-even with no subsample at all: below roughly n* = σ<sub>Y−f</sub><sup>2</sup>/b<sup>2</sup> (median 91, but 27 at the lower quartile and over 300 at the upper) a small subsample is worse than trusting the transferred model, and the formula, computable from a pilot, predicts where that happens within a factor of two. Precision is expensive either way: a group's share to ±3 points at 95% takes about 680 to 1,430 calibration respondents with the model (premarital sex 676, same-sex marriage 811, divorce law 1,252, papal infallibility 1,431) against 810 to 1,540 without; ±5 points takes about a third of that. For practice: rotate an item off a subgroup's ballot only if about 100 to 300 of that subgroup still get it; report the difference estimator with its Wald interval; and skip power tuning and recalibration, which buy nothing here. Caveats: subsamples are simple random draws from the respondents who were asked, while real GSS ballots are assigned within the clustered design, so true design effects would widen every interval; one boosted model and one seed; the full asked target stands in for the population, so the finite-population correction is about a known sample, not the US group. Next: a stratified or optimal (Neyman) allocation of a fixed calibration budget across groups, using each group's analytic n* and variance ratio, and the paper's section on the bound.

2026-11-12 · Milestone 6 (part): why does a transferred model miss a subgroup's level, and can the items the subgroup was asked correct it? (predictions written before the run; outcome below)

  • Why: milestone 1 found that models trained on one group rank another group's respondents almost as well but miss its level (third-generation to foreign-born Catholics: up to 11 points). Milestone 6 asks for a bound on a transferred model's error. The level error splits exactly: b<sub>j</sub> = E<sub>T</sub>[f<sub>j</sub>] − E<sub>T</sub>[Y<sub>j</sub>] = A<sub>j</sub> + B<sub>j</sub>, where A<sub>j</sub> = E<sub>T</sub>[f<sub>j</sub> − p<sub>S,j</sub>] is source misfit where the target lives (covariate shift; estimable with no target label as the importance-weighted source residual E<sub>S</sub>[r(X)(f<sub>j</sub> − Y<sub>j</sub>)], r = dT/dS from a domain classifier) and B<sub>j</sub> = E<sub>T</sub>[p<sub>S,j</sub> − p<sub>T,j</sub>] is conditional (concept) shift, which needs the target's answers to j. So |b<sub>j</sub>| ≤ |A<sub>j</sub>| + |B<sub>j</sub>|, with |A<sub>j</sub>| ≤ √(1 + χ<sup>2</sup>(T‖S)) ‖f<sub>j</sub> − p<sub>S,j</sub>‖<sub>S</sub>. The useful question is whether B<sub>j</sub> for the item a subgroup was not asked can be read off the items it was asked. experiments/m6_shift_bound/run.py fits gradient boosting (the milestone-1 model) on a source group for each of the 12 items (13 for Catholic-to-Catholic pairs) and imputes each item for a target group, over 20 source→target pairs: Catholic generation 3→1, 3→2, 1→3, non-Hispanic→Hispanic Catholics, all 12 ordered pairs of Catholic, Evangelical, Mainline and Black Protestant, and each tradition ≤2014→2016–24. For each held-out item j it compares estimators of the target's share that use only the target's covariates and its answers to the other items: E0 the transferred model as is; E1 the source share; IW, E0 minus the label-free Â<sub>j</sub>; E2, E0 shifted on the logit scale by the median of the other items' recalibration intercepts δ<sub>k</sub>; E3, shifted by Σρ<sub>jk</sub>δ<sub>k</sub>/Σ|ρ<sub>jk</sub>|, ρ<sub>jk</sub> the weighted correlation of the two items' predicted logits over the target's rows (label-free, so a negatively related item such as ever-divorced lends its shift with the sign flipped). Truth is the target's observed weighted share. A two-pair --quick run checked the code; these predictions were written before its output was read.
  • Predictions: (1) covariate shift explains little: median |Â<sub>j</sub>| under 1 point, and |Â<sub>j</sub>| < |B̂<sub>j</sub>| in at least 75% of cells, so IW is within 0.3 points of E0 on mean absolute error; (2) the naive transfer E0 misses by at least 3 points on average over cells and beats the source share E1; (3) E3 cuts E0's mean absolute error by at least 25%, with a pair-cluster bootstrap 95% interval for the paired difference excluding 0; (4) E2 helps less than E3, because shifts differ in sign across items (milestone 1: foreign-born Catholics are more certain of God but less likely to believe in an afterlife than the model expects); (5) the label-free correction size |E3 − E0| ranks cells by actual |b<sub>j</sub>| with Spearman ≥ 0.4; (6) the period pairs gain least, since the items that moved fastest (same-sex relations and marriage) moved apart from the rest. Refuted, with the reading that a subgroup's asked items carry its unasked item's level shift, if E3 is not better than E0 on mean absolute error (interval includes 0 or E3 worse).
  • Outcome (experiments/m6_shift_bound/results/cells.csv, summary.md; 241 cells over 20 pairs, about 9 minutes on 4 cores; intervals resample pairs, 2000 draws). Mean absolute error of the target's share in points: E0 5.59, E1 10.90, IW 5.25, E2 5.18, E3 5.24. Paired with E0: IW −0.33 (95% −0.69 to +0.04), E2 −0.41 (−0.96 to +0.16), E3 −0.35 (−0.74 to +0.01); E3 closer than E0 in 59% of cells. Median |b| 4.08 points, median |Â| 0.83, median |B̂| 3.45. (1) Held in its first two parts, missed narrowly in the third: median |Â| 0.83 points (< 1) and |Â| < |B̂| in 77% of cells (≥ 75%), but IW is 0.33 points better than E0, just past the 0.3 predicted; the gain is all in tradition pairs (5.73 → 5.18), where |Â| runs 1 to 4 points, and none in generation or period pairs (|Â| about 0.2). (2) Held: E0 misses by 5.59 points and beats the source share by 5.3 (3.9 to 6.8). (3) Refuted: E3 cuts E0's error by 6%, not 25%, and its interval reaches +0.01. (4) Refuted: E2 is no worse than E3 (E3 − E2 +0.06, −0.31 to +0.35); the signed weighting buys nothing. (5) Refuted: Spearman(|E3 − E0|, |b|) = 0.11; the correction's size does not say which cells are badly off, though its sign is right in 71% of cells. (6) Held: period pairs gain nothing (E3 6.93 vs E0 6.83). The refutation rule was met: a subgroup's answers to the items it was asked do not carry a usable estimate of the unasked item's level shift.
  • Reading: a transferred model's level error is mostly concept shift, item by item. Covariate shift, the part importance weighting can fix without any target label, is under a point for immigrant generations, Hispanic Catholics and later decades, and one to four points between traditions. The rest (median 3.45 points) is the target group answering this item differently from source respondents with the same profile, and that difference does not travel between items: the items whose shift E2/E3 fixed best were the sexual-morality cluster (premarital sex 5.5 → 3.3, same-sex relations 5.9 → 3.4, cohabitation 5.9 → 3.0), which move together, while abortion (4.6 → 6.2), divorce law (7.0 → 9.2) and adultery (2.3 → 3.5) got worse because their shifts run against the rest. For the bound this means |b<sub>j</sub>| ≤ |A<sub>j</sub>| + |B<sub>j</sub>| has a label-free first term and a second term that only target answers to j itself identify. For practice: before rotating an item off a subgroup's ballot, keep a calibration subsample that is asked it; the asked items' shifts are not a substitute. Caveats: one boosted model and one seed; the domain classifier's weights have a median effective sample of a third of the source; tradition pairs share respondents' years, so period and tradition shifts mix; cells within a pair are not independent (hence the pair bootstrap). Next: how large a target calibration subsample makes a difference estimator (E0 plus the subsample's mean residual; prediction-powered inference, Angelopoulos et al. 2023, Science, doi:10.1126/science.adi6000) beat the subsample's own mean, by item, which is the number a survey designer needs.

2026-11-12 · Milestone 3 (part): does giving the mixture the survey period let it recover a held-out ballot? (predictions written before the run; outcome below)

  • Why: the mixture entry (below) read the near-tie of MICE and the year-free DPMPM (4.66 and 4.81 points, both worse than interpolation's 3.43) as evidence that the barrier is missing period information, not the imputer's class. That reading is testable: give the same sampler the period. experiments/m3_period_mixture/run.py repeats m3_mixture exactly (Catholics, 13 items, 307 item-years, K = 80, 400 burn-in sweeps, m = 10 imputations 30 sweeps apart, the same seeds) with one added categorical variable observed for every respondent, the survey decade (--period decade), and as a second arm five-year bins (--period half). The exact year is still not a variable, so the held-out year's respondents can only borrow the item from same-period respondents in other years whose ballot asked it, through period-specific classes. Where an item was asked in no other year of the held-out year's period (same-sex marriage 1988 and 2004) the period carries nothing and the two models should agree. A one-item smoke run (same-sex marriage, 2022, 24 sweeps) was done to check the code; these predictions were written after it and before the full runs.
  • Predictions: (1) the decade mixture's mean absolute year-level error is at least 0.5 points below the year-free mixture's 4.81; (2) the gain is concentrated on the two items whose meaning moved: same-sex marriage below 12 points (year-free 20.3) and same-sex relations below 7 (10.5); (3) it still does not beat interpolation (3.43) on mean error, because a decade pools years that differ by up to 9 years of drift; (4) Rubin 95% coverage rises from 81.8% to at least 88% of item-years; (5) the five-year arm beats the decade arm on same-sex relations but not on mean error over all items (fewer same-period years per bin, so more noise). Refuted, and with it the reading that missing period information is what holds the joint model back, if the decade mixture is no better than the year-free mixture on mean error.
  • Outcome (experiments/m3_period_mixture/results/summary.md, period_mixture_decade.csv, period_mixture_half.csv; 307 item-years each, 258 and 235 CPU-minutes on 4 cores; occupied classes median 56 and 55 of K = 80). Mean absolute year-level error in points: decade mixture 3.55 (median 2.64; Rao-Blackwellised 3.54), five-year mixture 3.59 (3.53), against the year-free mixture 4.81, MICE 4.66, plain hgb 3.87, hgb_shift 3.38, decade mean 3.54, interpolation 3.43. Paired with the year-free mixture on the same item-years: decade −1.26 points (item-year bootstrap 95% interval −1.58 to −0.94), five-year −1.22 (−1.53 to −0.93); closer on 66% and 67% of item-years. (1) Held: 1.26 points better, well past the 0.5 predicted. (2) Held: same-sex marriage 11.5 points (year-free 20.3, under 12 predicted), same-sex relations 5.7 (10.5, under 7 predicted). (3) Held: it does not beat interpolation (3.55 vs 3.43; closer on 50% of item-years). (4) Held: Rubin 95% coverage 91.5% (year-free 81.8%), same-sex marriage 3 of 5 years (0 of 5), same-sex relations 73% (40%), at about the same half-width (7.5 vs 7.6 points); Winkler score 25.0 against 44.9. (5) Refuted in its first half: the five-year arm is not better on same-sex relations (5.9 vs 5.7); it is, as predicted, not better on mean error (3.59 vs 3.55). The refutation rule was not met. Averaged with interpolation, the decade mixture gives 3.15 points, the lowest point error in the programme (earlier best combo 3.23); with the drift interval it covers 95.1% (worst item 79%) at a Winkler score of 21.5, level with combo (21.1).
  • Reading: one variable, the survey period, takes a quarter off the mixture's error and nearly closes the gap to the neighbour baselines, which confirms the mixture entry's reading that missing period information, not the imputer's class, held both principled imputers back. But the period-aware mixture lands on the decade mean (3.55 vs 3.54), not below it: as an estimator of a wholly unseen year's share, it has learned about what the period's other ballots already say. The other answers rank respondents well (milestone 1) but add nothing to the level beyond the period. The classes do not split by period: a median 2% of the held-out year's respondents sit in classes mostly drawn from their own decade, so the period works by tilting assignment among mixed classes. Finer bins buy nothing (five-year and decade within 0.04 points), and the six decade item-years with no other asked year in their period (same-sex marriage 1988 and 2004 among them) stay bad (10.2 points). For practice: a joint imputer for a rotated GSS item should carry the period; the honest product is still neighbour-anchored (now combo_pmix, 3.15) with the drift interval. Caveats: one chain per item-year; the mixture is unweighted; demographics binned; Kish effective n, not the GSS design; same_period_n counts only held-out-eligible years (asked of at least 150). Next: milestone 5's local-model baseline, or a mixture with a smooth period effect (neighbouring decades sharing strength) for the end-of-run years.

2026-11-11 · Milestone 3 (part): does Bayesian mixture imputation (Si and Reiter's DPMPM) recover a held-out ballot? (predictions written before the run; outcome below)

  • Why: the proposal names two principled baselines for milestone 3, MICE (2026-11-10) and the Dirichlet-process mixture of product multinomials of Si and Reiter (2013), which earlier entries deferred because the GSS host refused the proxy. It answered this session (checksums match data/SHA256SUMS). src/dpmpm.py is a numpy blocked Gibbs sampler for the truncated mixture (Dunson and Xing 2009; Ishwaran and James 2001): all variables categorical, missing cells ignored under MAR, so no separate feature imputation. experiments/m3_mixture/run.py repeats the held-out-year design (Catholics, 13 items, item-years asked of at least 150, year not a variable) with the raw item codes and binned demographics, the target as one more variable missing for the held-out year, K = 80 classes, 400 burn-in sweeps, m = 10 imputations 30 sweeps apart, and Rubin's rules as in the MICE entry. A timing run (weekly attendance, 2010) found K = 30 fully occupied (30 of 30, alpha about 10), so K was raised to 80 (about 64 occupied); the 2010 share moved by under 1.2 points across K = 30, 50, 80. These predictions were written after that timing run and before the full run.
  • Predictions: (1) the mixture's mean absolute year-level error is between 3.8 and 5.0 points: better than MICE (4.66) because the classes carry interactions a main-effects logistic model lacks, but not better than plain hgb (3.87); (2) it does not beat interpolation (3.43); (3) its Rubin 95% intervals cover the observed share in under 85% of item-years and in at most 40% of same-sex-marriage years, the item still worst with error above 10 points; (4) the drift-aware conformal interval of the last entry, put around the mixture, covers at least 90% overall with every item at least 70%. Refuted, and with it the reading that no year-free model beats carrying the neighbours, if the mixture beats interpolation on mean error.
  • Outcome (experiments/m3_mixture/results/mixture.csv, summary.md; 307 item-years, 13 items, 211 CPU-minutes on 4 cores; occupied classes median 53, max 69 of K = 80, so the truncation did not bind). Mean absolute year-level error in points: dpmpm 4.81 (median 3.49), its Rao-Blackwellised share 4.76, against MICE logreg 4.66, plain logit 4.06, plain hgb 3.87, hgb_shift 3.38, decade mean 3.54, interpolation 3.43. (1) Refuted in part: the error is inside the predicted 3.8 to 5.0 band, but the mixture is not better than MICE (4.81 vs 4.66; closer on 48% of item-years) and is worse than plain hgb (closer on 37%). (2) Held: closer than interpolation on 43% of item-years, 1.4 points worse on average. (3) Held: Rubin 95% intervals cover 81.8% of item-years and 0 of 5 same-sex-marriage years, which is the worst item at 20.3 points (interpolation 5.8); same-sex relations 10.5 (interpolation 2.9), covered in 40% of years. (4) Half held: the drift-aware conformal interval around the mixture covers 93.8% overall, but its worst item is same-sex relations at 63%, under the 70% predicted. Averaged with interpolation the mixture helps less than hgb_shift does (3.57 vs 3.23 points; drift interval 95.1% with worst item 80%, Winkler 25.9 vs 21.1). The refutation rule was not met.
  • Reading: the two textbook principled imputers, chained equations and a Bayesian latent-class mixture, are built on opposite assumptions (linear conditionals vs a joint model with every interaction carried by classes), yet they land within 0.15 points of each other on a held-out ballot (4.66 and 4.81), both about 1.3 points worse than carrying the neighbouring years' shares and with the same over-confident intervals (82 to 83% coverage at a nominal 95%, near zero on same-sex marriage). That agreement is the result: the barrier is not the imputation model's class but the missing period information. Both pool every year into one joint distribution, so the held-out year's respondents are assigned the pooled relation between their other answers and the item, and on the two items whose meaning moved (same-sex relations and marriage) that relation is the wrong one. Milestone 3 now has its full baseline set: on this test no year-free imputer, simple or principled, beats the neighbours, and the honest product remains the neighbour-anchored combo with the drift interval (2026-11-11, entry below). Caveats: one chain per item-year with no convergence diagnostic beyond the occupied-class count; the mixture is unweighted (as in Si and Reiter) while the shares are weighted; demographics were binned into 5 or 6 levels; Kish effective n, not the GSS design. Next: the step in milestone 3 these baselines leave open is a period-aware joint model (for example the mixture with decade as a variable, so classes can be period-specific), then milestone 5's local-model baseline.

2026-11-11 · Milestone 3 (part): can an interval for an unseen ballot's share be made honest by pricing in period drift? (predictions written before the run; outcome below)

  • Why: the MICE entry (below) found Rubin 95% intervals cover the held-out year's observed share in 83% of item-years overall but 20 to 40% on the two items whose trend broke, because they carry within-model uncertainty only. The GSS host (gss.norc.org) refused this session's proxy (403), so this step uses only the committed per-item-year CSVs (m3_period_shift, m3_mice): no microdata is needed to recalibrate an interval around an imputed share already computed. experiments/m3_drift_intervals/run.py builds leave-one-item-out split-conformal intervals (Vovk, Gammerman and Shafer 2005; Lei et al. 2018) around three point imputations (interp, hgb_shift, mice_logreg) and their average combo = (interp + hgb_shift) / 2. For the item being scored, the 95% quantile of the scores of the other 12 items' item-years sets the width. Three scores: const (|error|, a fixed width), samp (|error| / s, s the binomial sampling sd of the year's share at its n), and drift (|error| / sqrt(s<sup>2</sup> + d<sup>2</sup>)), where d is a drift scale from the item's other asked years only: the per-year change between the two neighbouring asked years (both sides if the year is interior, the two nearest on one side if not) times the distance to the nearest asked year.
  • Predictions: (1) drift intervals around hgb_shift cover 90 to 97% of item-years overall; (2) their worst single-item coverage is at least 70% (Rubin MICE: 20% on same-sex marriage); (3) at matched overall coverage drift has a smaller median half-width than const; (4) samp alone under-covers the fast-moving items (below 60% on same-sex marriage or same-sex relations), as Rubin did; (5) combo has a lower mean absolute error than both its parts (interp 3.43, hgb_shift 3.38). Refuted, as a way to make intervals honest where opinion moved, if drift worst-item coverage is below 60%.
  • Outcome (experiments/m3_drift_intervals/results/summary.md, intervals.csv; 307 item-years, 13 items, under 5 s, no microdata). Around hgb_shift, leave-one-item-out 95% intervals cover: const 94.1% (worst item 60%, same-sex marriage), samp 95.1% (worst 60%), drift 95.1% (worst 83%, divorce law; same-sex marriage 5 of 5 years, same-sex relations 87%). Median half-widths 9.4, 8.2 and 7.6 points; mean interval (Winkler) scores 30.1, 28.5 and 22.2. For comparison, Rubin's MICE intervals cover 82.7% (worst 20%) and the hgb_shift bootstrap 76.2% (worst 40%). (1) Held: 95.1%. (2) Held: worst item 83%. (3) Held: drift is narrower than const (7.6 vs 9.4 points) at slightly higher coverage, and has the best interval score. (4) Refuted narrowly for hgb_shift: samp covers same-sex marriage in exactly 60% of years (3 of 5) and same-sex relations in 87%, not below 60%; around mice_logreg it is 20% and 50%, as predicted. (5) Held: combo mean error is 3.23 points, below interp (3.43) and hgb_shift (3.38), and combo + drift has the lowest interval score of all (21.1). The refutation rule was not met. The pattern holds at 80% and 90% nominal levels (drift worst-item 66% and 72%, against 40% and 59% for const).
  • Reading: the fix for MICE's over-confident intervals is not more imputations but a width that grows where the item's neighbours were moving. A conformal interval calibrated on the other items, with the score scaled by sampling noise and a drift scale read off the neighbouring asked years, reaches nominal coverage overall and on the fast-moving items while being narrower in the typical year. So for a survey analyst the usable product is a point imputation (the neighbour average blended with a shifted model, combo) plus this interval, about ±7 points for a typical Catholic item-year: honest, and wide. The drift scale itself is a weak predictor of the error (Spearman 0.13); it earns its keep in the tail, the few years between asked years that straddle a break. Caveats: conformal validity here assumes the 13 items are exchangeable, which they are not (same-sex marriage has 5 years, so its 100% is 5 of 5); scores within an item are correlated across years; s ignores weights and the GSS design effect; hgb_shift's neighbour offsets are in-sample (2026-11-08); and none of this helps an item never asked, which has no neighbours. Next: the mixture baseline (Si and Reiter 2013) when the GSS host is reachable again, and the same intervals around it.

2026-11-10 · Milestone 3 (part): does chained-equation multiple imputation (MICE) recover a held-out ballot, and are its intervals honest? (predictions written before the run; outcome below)

  • Why: the proposal names MICE and mixture imputation as milestone 3's principled baselines, and the three period-aware single models (2026-11-08, 2026-11-09) all stopped near interpolation. MICE adds two things the earlier variants lack: missing features are imputed jointly with the target instead of median-filled or split around, and Rubin's rules give an interval for the imputed share. experiments/m3_mice/run.py repeats the held-out-year design (Catholics, 13 items, item-years asked of at least 150 respondents, year not a feature) with m = 5 chained-equation imputations (sklearn IterativeImputer, BayesianRidge, posterior draws, 5 rounds) and two target models from the same completed data: mice_norm (the chain's own linear draw, clipped) and mice_logreg (logistic regression on a bootstrap resample, Bernoulli draws; mice's "logreg.boot"). A one-item timing run (weekly attendance, 2010, m = 2) was done to size the job; these predictions were written after it and before the full run.
  • Predictions: (1) mice_logreg mean absolute year-level error is within 0.3 points of plain logit (4.06) and does not beat plain hgb (3.87): imputing features jointly does not tell the model anything about the unseen year; (2) mice_norm is worse than mice_logreg (a linear-probability draw, clipped, mis-states shares near 0 or 1); (3) the Rubin 95% interval covers the year's observed share in under 80% of item-years for both, since it carries within-model uncertainty only, not period drift; (4) same-sex marriage is still the worst item, with error above 10 points. Refuted, and the reading of the last three entries with it, if mice_logreg beats interpolation (3.43) on mean error.
  • Outcome (experiments/m3_mice/results/mice.csv, summary.md; 13 items, 307 item-years, m = 5, about 25 min on 4 cores). Mean absolute year-level error in points: mice_logreg 4.66 (median 3.46), mice_norm 7.18, against plain logit 4.06, plain hgb 3.87, hgb_shift 3.38, hgb_w5 3.44, decade mean 3.54, interpolation 3.43. (1) Refuted in the worse direction: mice_logreg is 0.6 points worse than plain logit, not within 0.3, and does not beat hgb (closer than hgb on 39% of item-years, than interpolation on 42%). (2) Held: the clipped linear draw is far worse (7.2). (3) Half held: the Rubin 95% interval covers the observed share in 39.7% of item-years for mice_norm but 82.7% for mice_logreg (80% predicted as the ceiling), with a median half-width of 7.4 points. Both are below the nominal 95%, and coverage collapses where opinion moved: same-sex relations 40%, same-sex marriage 20% (1 of 5 years). (4) Held: same-sex marriage is the worst item (15.8 points; interpolation 5.8). The refutation rule was not met: MICE does not beat interpolation, so the reading of the last three entries stands.
  • Reading: the textbook multiple-imputation workflow is the worst of the model-based ways tried so far to recover a wholly unseen ballot's share. Imputing the missing features jointly did not help, and on same-sex relations (10.0 vs 6.8 for plain logit) and adultery (5.6 vs 3.9) it hurt; one possible cause, not tested, is that features imputed from the same pooled linear chain shrink the held-out year's profiles toward the all-years average. The useful finding is about the intervals: MICE's Rubin intervals look reasonable on average (83% coverage at a nominal 95%) only because the Bernoulli draws make them wide (median ±7.4 points); they cover 20 to 40% of the years on the two items whose trend broke, so an analyst trusting them would be told a 1990s same-sex-relations share was known to within a few points when it was off by ten. An interval that includes period drift (for example, the spread of the neighbouring years' shares) is the natural next step, together with the mixture baseline (Si and Reiter 2013). Caveats: one seed per imputation; m = 5; the chain is linear (BayesianRidge) for every feature including one-hot categoricals; the within-imputation variance uses a Kish effective n, not the GSS design; the mixture baseline is still not run.

2026-11-09 · Milestone 3 (part): does fitting on the nearest years only fix the held-out-year bias? (predictions written before the run; outcome below)

  • Why: the period-shift check (entry below) left the best variant, hgb_shift (3.38 points), at parity with interpolation (3.43), and the level shift did not close the gap on fast-moving items, which suggests the relation between other answers and the item changed, not only its level. experiments/m3_local_window/run.py fits the year-free logit and hgb only on training years within 5 or 10 years of the held-out year (fallback to all other years when the window has under 600 respondents or one class). Same 13 items, 307 item-years, 200 bootstrap draws and baselines. The predictions were written after the job started but before I opened any output.
  • Predictions: (1) the best window variant has a lower mean absolute year-level error than plain hgb (3.87) by at least 10%; (2) it does not beat interpolation (3.43) by more than 0.3 points; (3) the 5-year window is better than the 10-year one on the fast-moving items (same-sex marriage, cohabitation) and worse on the slow ones (attendance, premarital sex), where less data costs more than drift; (4) at least 78% of year intervals contain zero for the best variant. Refuted if the best window variant is no better than plain hgb on mean error.
  • Outcome (experiments/m3_local_window/results/local_window.csv; 13 items, 307 item-years, 200 bootstrap draws, about 15 min; the window fell back to all other years in 11 item-years at W = 5 and 5 at W = 10). Mean absolute year-level error in points: logit_w5 4.84, logit_w10 4.38, hgb_w5 3.44, hgb_w10 3.48, against plain hgb 3.87, hgb_shift 3.38, decade mean 3.54, interpolation 3.43. (1) Held: hgb_w5 cuts plain hgb error by 11%. (2) Held: it matches interpolation (3.44 vs 3.43; closer on 56% of item-years) and does not beat it by 0.3. (3) Refuted: the 5-year window is not systematically worse on slow items (attendance 2.5 vs 2.8 for W = 10; premarital sex 2.9 vs 2.6 is the one case that is) and ties on same-sex marriage (16.7 both). (4) Refuted narrowly: 77.5% of hgb_w5 year intervals contain zero (78% predicted); 76% for hgb_w10. Under the stated rule the window helps (3.44 < 3.87), so the hypothesis that period-local fitting helps is not refuted, but it only reaches parity with interpolation and the shift, as the level shift did.
  • Reading: three different ways to make a model period-aware (a linear year term, a neighbour-fitted level shift, a local window) all stop at roughly 3.4 to 3.7 points, the same as carrying the neighbouring ballots' observed shares. A boosted model on other answers adds nothing about a wholly unseen ballot beyond what its neighbours already say. The exception is same-sex marriage, 16.7 points for both windows against 5.8 for interpolation: the item moved so fast that even a 5-year window straddles the break, and the logit windows are worse than hgb everywhere (4.4 to 4.8). So milestone 3's remaining question is no longer 'which period fix' but whether MICE or mixture imputation does better on items that were never asked; the held-out-year test cannot separate them from interpolation. Caveats: one seed; the bootstrap ignores GSS clustering; windows that are small fall back to all years (11 item-years), which blurs the W = 5 result at the ends of runs; same-sex marriage dominates the spread.

2026-11-08 · Milestone 3 (part): does a period term fix the held-out-year bias? (predictions written before the run; outcome below)

  • Why: the held-out-year check (entry below) left hgb no better than carrying neighbouring years' shares, worst where opinion moved fast, since with year removed the model cannot learn a trend. experiments/m3_period_shift/run.py repeats it with three fixes: logit_year (linear year term), and logit_shift / hgb_shift, a logit offset fitted so the year-free model reproduces the observed share in the nearest earlier and later asked years, interpolated in year. Same items, years, bootstrap (200 draws) and baselines. The hgb neighbour fit is in-sample, so its offsets are optimistic (disclosed).
  • Predictions: (1) the shift variants cut mean absolute year-level error of the plain models (hgb 3.9 points) by at least a quarter; (2) logit_shift or hgb_shift matches or beats interpolation (3.4) on mean error, but does not beat it by more than 0.5 points, since both lean on the same two neighbours; (3) logit_year helps the fast-moving items (same-sex marriage, cohabitation) but is worse than the shift on end-of-run years; (4) the share of year intervals containing zero rises from 69% to at least 80% for the best variant. Refuted if the best shift variant is no better than plain hgb on mean error.
  • Outcome (experiments/m3_period_shift/results/period_shift.csv; same 13 items and 307 item-years, 200 bootstrap draws, about 20 min). Mean absolute year-level error in points: plain logit 4.06, plain hgb 3.87, logit_year 3.70, logit_shift 3.84, hgb_shift 3.38, decade mean 3.54, interpolation 3.43. (1) Refuted: the best shift cuts hgb error by 13%, not a quarter. (2) Held, barely: hgb_shift matches interpolation (3.38 vs 3.43; lower on 56% of item-years) and does not beat it by 0.5. (3) Refuted: logit_year is no better on the fast items (same-sex marriage 17.9 vs 17.3 for plain hgb, cohabitation 7.8 vs 6.9) and about equal to hgb_shift at the ends of runs (6.25 vs 6.34 points). (4) Refuted: 76% of hgb_shift year intervals contain zero (logit_year 74%, plain hgb 69%). Under the stated rule the shift is better than plain hgb on mean error, so the programme's hypothesis that a level shift helps is not refuted, but the gain is small.
  • Reading: a level shift fitted on the neighbouring ballots buys a model parity with interpolation and no more, so imputing the share of a wholly unseen ballot gains nothing over carrying the neighbours' observed shares. Where opinion moved fast the shift does not close the gap: same-sex marriage stays at 15.6 points (interpolation 5.8) and same-sex relations 4.4 (2.9), which means the relation between a respondent's other answers and the item itself changed, not only the level. The next step is a period-by-feature interaction (or a model fitted on the nearest years only), and the MICE and mixture baselines. Caveats: one seed; the hgb neighbour offsets are in-sample and so optimistic, yet hgb_shift is still only at parity; the bootstrap ignores GSS clustering; interpolation needs asked neighbours that an item never asked does not have. The m3_year_holdout/run.py import of the milestone 1 module was changed to a path-based load so both scripts can be imported together; its results are unchanged.

2026-11-08 · Milestone 3 (part): does the imputed share match a ballot the model never saw? (predictions written before the run; outcome below)

  • Why: milestone 3 asks whether imputed shares match the shares from ballots that asked the item. The out-of-fold check of 2026-11-07 still let the model see the same survey year. experiments/m3_year_holdout/run.py holds out a whole survey year for each Catholic item (at least 150 respondents asked), fits on all other years without year as a feature, and compares the weighted mean prediction with that year's observed share (200 respondent-bootstrap draws). Baselines for the same share: the mean of the other years in the decade, and linear interpolation between the nearest earlier and later asked years. Models: hgb and L2 logit. The predictions were written after the job started but before I opened any output.
  • Predictions: (1) the median absolute year-level bias of hgb is under 3 points; (2) at least 80% of year intervals contain zero (wider than the 94% of the out-of-fold check, because a year shifts the level); (3) hgb has a lower mean absolute error than the decade mean on most item-years, but not than interpolation on the slow-moving items (attendance, premarital sex), where interpolation is hard to beat; (4) hgb is worst in years at the ends of an item's run (first and last year), where interpolation has only one neighbour.
  • Outcome (experiments/m3_year_holdout/results/year_holdout.csv; 13 items, 307 item-years, 200 bootstrap draws, about 15 min). (1) Held: median absolute year-level bias of hgb is 2.8 points (mean 3.9; logit median 2.6, mean 4.1). (2) Refuted: only 69% of year intervals contain zero (80% predicted). (3) Refuted: hgb has a lower error than the decade mean in only 47% of item-years and than interpolation in 50%; mean absolute error is 3.9 points for hgb, 3.5 for the decade mean and 3.4 for interpolation. (4) Held: errors at the first and last year of an item's run are larger for both (hgb 6.9, interpolation 5.4 points) than in the interior (3.6 and 3.2).
  • Reading: for the share of a whole unseen ballot, a model fitted on other years' answers is no better than simply carrying the neighbouring years' observed shares. It wins on certainty about God (1.9 vs 3.0 points for interpolation), and loses badly where opinion moved fast: opposing same-sex marriage is off by 17.3 points for hgb against 5.8 for interpolation, and cohabitation and same-sex relations by 6.7 to 6.9 against 2.9. With year removed the model cannot learn a secular trend, so its level is stuck at the pooled average; adding a period term, or fitting a shift on neighbouring years, is the obvious next step. The out-of-fold check (2026-11-07) looked better because the model saw the same year. Caveats: one seed, a bootstrap that ignores GSS clustering, the two baselines use the held-out items' own neighbouring ballots (not available for an item never asked), and no MICE or mixture baseline yet.

2026-11-07 · Milestone 2 (part): is the imputed share right in each decade and generation? (predictions written before the run; outcome below)

  • Why: milestone 3 asks whether imputed shares match the shares from ballots that asked the item. Out-of-fold predictions are the same test in a cleaner form: every respondent's answer is predicted from a model that never saw them. experiments/m2_calibration/run.py compares the weighted mean out-of-fold hgb prediction with the weighted observed share in each decade x immigrant-generation cell (at least 100 respondents), with 200 bootstrap draws, and computes a weighted expected calibration error (10 equal-count bins) per Catholic target.
  • Predictions: (1) the median absolute cell bias is under 2 points; (2) at least 90% of cell intervals contain zero; (3) the largest biases are in the first-generation cells (small, shifted levels); (4) ECE is under 0.03 on every target.
  • Outcome (experiments/m2_calibration/results/cell_bias.csv, ece.csv; 13 targets, 180 decade x generation cells, seed 0, about 1 minute). (1) Held: median absolute cell bias 1.2 points. (2) Held: 170 of 180 intervals (94%) contain zero; at a 95% level about 9 would miss by chance, and 10 did. (3) Half held: the median bias is largest for first-generation cells (1.7 points, vs 1.3 second and 1.0 third-plus), but the two largest single misses are second-generation 1970s divorce-harder (-8.4 points, interval excludes zero) and first-generation 1980s sex-before-marriage (-6.7). (4) Refuted for one item: ECE is under 0.03 for 12 of 13 targets (0.006 to 0.026); cohabitation is 0.041 (n = 1,489, the smallest ballot), and same-sex marriage is 0.026 on 1,869.
  • Reading: out-of-fold imputed shares match the observed shares within about 1 point in a typical decade x generation cell, and the cells that miss are as often second-generation as first. The two small-ballot items (cohabitation, same-sex marriage) are the least calibrated. Caveats: one fold seed; the cells' bootstrap ignores GSS clustering and strata; these are out-of-fold checks on asked respondents, not yet a comparison against a ballot that asked the item while the model never saw that ballot (milestone 3).

2026-11-07 · Milestone 2 (part): Hispanic vs non-Hispanic first-generation Catholics (predictions written before the run; outcome below)

  • Why: H3 found that a model fitted on third-plus-generation Catholics ranks foreign-born Catholics almost as well as the pooled model. Foreign-born Catholics are mostly Hispanic, so the pooled result may hide a split. experiments/m2_hispanic/run.py fits hgb on generation-3 Catholics and scores first-generation Hispanic and non-Hispanic Catholics separately against the pooled out-of-fold predictions (seed 0, 200 bootstrap draws).
  • Predictions: (1) the AUC loss is within 0.03 for both groups on most items; (2) the loss is larger for Hispanic than non-Hispanic immigrants on most items (the generation-3 training set is mostly non-Hispanic white, so shifted conditionals bite where it differs most); (3) level gaps exceed 5 points on more items for Hispanic immigrants. Refuted if the Hispanic loss is no larger than the non-Hispanic loss on at least half the items.
  • Outcome: experiments/m2_hispanic/results/hispanic_transfer.csv (first-generation Catholics: 663 to 1,174 Hispanic and 208 to 493 non-Hispanic per item on 11 items; two small items, same-sex marriage and cohabitation, are scored for Hispanic immigrants only). Loss = pooled out-of-fold AUC minus generation-3-model AUC on the same rows. (1) Held: the loss is within 0.03 on every Hispanic item and on 10 of 11 non-Hispanic items; the exception is ever-divorced (0.057, interval −0.002 to 0.127). (2) Not supported: Hispanic loss is larger on 6 of 11 paired items, no systematic difference, and every paired interval overlaps zero except divorce-law for non-Hispanic immigrants, where the generation-3 model is better (−0.065). Under the stated rule (refuted if no larger on at least half), 6 of 11 does not refute it, but it is not a result either. (3) Weakly held: level gaps exceed 5 points on 6 of 13 Hispanic items and 4 of 11 non-Hispanic items. The large level misses are for Hispanic immigrants on certainty about God (−13 points), afterlife (+12.5), ever-divorced (+13) and divorce law (+8).
  • A new observation, not predicted: absolute AUC is lower for Hispanic immigrants on the practice and doctrine items (weekly Mass 0.77 vs 0.87, daily prayer 0.71 vs 0.80, certainty about God 0.76 vs 0.82), whichever model is used. The other answers rank non-Hispanic immigrants better, so imputation is less accurate for the largest immigrant group even before any transfer.
  • Reading: the pooled H3 result does not hide a Hispanic-specific transfer failure in ranking. Levels do move more for Hispanic immigrants on the God, afterlife and divorce items. One fold seed, 200 bootstrap draws, and the two groups differ in size, so intervals are wide. Still not done: the check of weighted shares against NORC's Data Explorer, calibration plots and design-based variance.

2026-11-06 · Milestone 2 (part): CV-seed stability and a politics ablation, Catholics (predictions written before the run; outcome below)

  • Why: milestone 1 used one fold seed, so its AUCs carry no split-to-split spread, and the permutation-importance table suggests politics matter but never removes them. experiments/m2_seeds_ablation/run.py: Catholic respondents, 14 targets, logit and hgb, 5-fold weighted CV with seeds 0 to 2, on the full feature set and with polviews and partyid removed.
  • Predictions: (1) the seed-to-seed range of AUC is under 0.01 for every target (samples are 2,000 to 18,000); (2) removing politics lowers hgb AUC by under 0.02 on the sexual-morality items (the other answers already carry the ideology); (3) the largest loss is on abortion or same-sex marriage items, not on attendance; (4) the logit-vs-hgb gap of milestone 1 (+0.003 median) stays under 0.01 across seeds.
  • Outcome (experiments/m2_seeds_ablation/results/seeds_ablation.csv; 13 targets, 3 fold seeds, about 25 min on 4 cores; the 14th target, afterlife-vs-other, was not defined for Catholics). Mean AUC over seeds, full vs no-politics hgb: attendance 0.861 / 0.861, premarital sex 0.852 / 0.851, same-sex relations 0.837 / 0.833, same-sex marriage 0.883 / 0.873, abortion 0.750 / 0.746, ever-divorced 0.710 / 0.709.
  • Predictions: (1) refuted narrowly: the largest seed range is 0.013 (cohabitation, no-politics hgb), 0.0095 with politics; all other cells are under 0.01. (2) held: the largest politics loss on any item is 0.010 (hgb) and 0.013 (logit), both on same-sex marriage. (3) held: that item has the largest loss, attendance has none. (4) mostly held: the median hgb minus logit gap is +0.002, under 0.01 on 10 of 13 items; the exceptions are ever-divorced (+0.035), cohabitation (+0.018) and adultery (+0.010), the same nonlinear-age items as milestone 1.
  • Reading: milestone 1's AUCs are stable to within about 0.01 across fold seeds, and politics (self-placement and party) carry almost nothing beyond the other answers except on same-sex marriage. This is Catholics only, 3 seeds, and seed ranges are from CV splits of one sample, not sampling error of the survey.
  • Not done here: Hispanic split, the check of weighted shares against NORC's Data Explorer and research/catholic-demography, calibration plots, design-based variance.

2026-11-03 · Milestone 1 results: other answers predict unasked items well; associations transfer, levels do not

  • Ran experiments/m1_imputation/run.py on the GSS 1972–2024 R3a (fit and test about 30 min on 2 cores). Tables are in experiments/m1_imputation/results/summary.md, and summarise.py rebuilds them from the committed CSVs.
  • A first full run crashed in the temporal step. Some items were never on the same ballot before 2014, which left an all-missing feature column. Such columns are now dropped per training split, and the whole run was redone.
  • Headline. Across 49 tradition × item cells, a model given a respondent's other answers beats the decade prior by a median 0.21 AUC. Its log loss is lower in 44 of the 49 (paired 95% interval below zero). - Among Catholics, boosting reaches 0.86 for weekly Mass attendance, 0.85 for "sex before marriage always wrong", 0.84 for same-sex relations, 0.88 for opposing same-sex marriage, 0.82 for cohabitation, 0.75 for abortion, 0.69 for divorce law and 0.71 for ever-divorced.
  • H1: mostly supported. The four sexual-morality and marriage-belief items clear 0.75. Adultery (0.73) does not: almost everyone (77%) says it is always wrong, so there is little to rank. Divorce law stays under 0.70 as predicted. Ever-divorced reaches 0.71 with boosting (logistic 0.67), which breaks the "< 0.70" half of H1 for that model.
  • H2: supported. The median boosting − logistic gap is +0.003 AUC. Only 4 of 49 cells have an interval wholly above +0.01, and 3 of those are ever-divorced, where age and exposure act nonlinearly. Logistic regression is never worse by more than 0.01.
  • H3: supported. A model fitted only on third-plus-generation Catholics ranks foreign-born Catholics within 0.03 AUC of the pooled model on 12 of 13 items (ever-divorced −0.038). Its levels are wrong by more than 5 points on 5 of 13: - It under-predicts certainty about God by 11 points and opposition to same-sex marriage by 8. - It over-predicts belief in an afterlife by 10, support for harder divorce by 7 and ever-divorced by 8. - So the association structure largely travels to immigrants, but group levels do not. An imputation model must carry generation (and later country of origin) as a level term.
  • H4: refuted. Scored on Protestants, a Catholic-fitted model loses a mean 0.011 AUC on Evangelicals, 0.005 on Mainline and 0.007 on Black Protestants. Black Protestants are not the worst. Again levels are the problem: on Evangelicals the Catholic model under-predicts "sex before marriage always wrong" by 17 points and averages 6.7 points low.
  • Descriptive (Catholics, weighted share, by decade and generation; shares_catholic_decade_generation.csv): - Weekly Mass attendance fell from 41–48% in the 1970s to 14–19% in the 2020s, in every generation. - In the 2020s, foreign-born Catholics are the most likely to oppose abortion for any reason (62% vs 47% third-plus generation) and to hold papal infallibility (56% vs 37%).
  • Orthodox Christians: only 27–205 GSS respondents per item, below the 300 floor, so no models were fitted. Global and Orthodox coverage waits on EVS/WVS/ISSP access (milestone 4).
  • Not yet checked against a published figure. This is milestone 2's first task: compare the weighted shares with NORC's GSS Data Explorer and with research/catholic-demography once it lands. Still to do: fold seeds (one seed so far), calibration plots, design-based (PSU and strata) variance.

2026-11-03 · Session 1, plan for milestone 1 (written before the run)

  • Data: GSS 1972–2024 cumulative cross-section, release 3a (NORC; data/download.sh, checksums in data/SHA256SUMS match the copy research/marriage-divorce-evidence records). 75,699 respondents; Catholic (relig 2), Orthodox (relig 10) and the three Protestant traditions of Steensland et al. (reltrad 1–3).
  • Targets: 12 binary items coded as the answer consistent with Catholic teaching (Catechism paragraph in src/gss.py) or a practice level, plus ever-divorced among the ever-married and, for Catholics, papal infallibility.
  • The GSS asks many items on only some ballots, so most "missing" answers are missing by design (at random). That makes it a fair test of imputation: predict an item a respondent was not asked from the ones they were.
  • Planned analysis: 5-fold weighted cross-validation within each tradition. Models are the decade prior, L2 logistic regression and gradient boosting. Metrics are weighted AUC, log loss and Brier, with paired respondent bootstrap intervals (300 draws). Then a temporal hold-out (train ≤2014, test 2016–2024), an immigrant-generation transfer, a cross-tradition transfer and permutation importance.
  • H1. Other answers predict the sexual-morality items well beyond the prior (AUC ≥ 0.75 among Catholics). Divorce law and ever-divorced are much harder (AUC < 0.70). Refuted if sexual-morality AUCs are below 0.75, or divorce-law AUC reaches them.
  • H2. Gradient boosting beats logistic regression by less than 0.01 AUC on most items, as flexibility did not help in marriage-divorce-evidence. Refuted if the bootstrap interval for the boosting minus logistic AUC gap lies above 0.01 on most items.
  • H3. A model fitted on third-plus-generation Catholics ranks first-generation (foreign-born) Catholics almost as well as the pooled model does (AUC loss ≤ 0.03), but its level is off: the mean prediction misses the first-generation share by more than 5 points on at least a third of items. Refuted if the AUC loss exceeds 0.03 on most items, or levels transfer within 5 points everywhere.
  • H4. A Catholic-fitted model ranks Evangelical and Mainline Protestants within 0.03 AUC of their own models. It ranks Black Protestants worse, because party and ideology carry less information about sexual morality there. Refuted if the Black Protestant loss is no larger than the other two.

Paper, code and results

More in History & human sciences

All 4 History & human sciences projects →

Other research topics

Proper scoring & elicitation28Effort & peer prediction9Verification games & slashing19Detection & tolerances15Markets as routers13Prediction-market design9Paying contributors8Decentralised training systems23Digital twins & sim-to-real164LLM chip design loops40Robot manufacturing loops53Ambient AI assistant46Self-replicating manufacturing34Simulation-first design64Engineered clothing45Task-specific AI accelerators40Algorithmic chip-design loops42Open-source purpose-built robots50The software-engineering loop43Adversarial human imitation41Open-source residential construction41Autonomous multi-angle capture35Sensing, imaging and 3D33Energy supply and dense energy storage41Robot societies among people32Agentic co-design of low-cost robots28Laundry-hanging robots26Machine learning on qualitative data23Robots on the building site28Civilizations & group psychology3History & human sciences4

Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.