Substrate/research/meaning-crisis-hypotheses-2026-04/METHODOLOGY.md

Methodology

Research Project: Meaning Crisis — Causal Hypotheses (PR-00001) Date: 2026-04-22

---

Research Design

Two-phase process: (1) hypothesis generation via BeCreative, (2) hypothesis evaluation via Science FullCycle protocol.

Research Duration: Single session, 2026-04-22 Substrate Datasets Consulted: 6 (DE-World-Values, DE-Mental-Health, DE-Church-Exits, DE-Social-Isolation, DE-Platform-Media, DE-Epistemic-Competence) External sources queried: None — evidence exclusively from curated Substrate datasets

---

Phase 1: Hypothesis Generation — BeCreative (Verbalized Sampling)

Protocol

BeCreative uses Verbalized Sampling: generate N candidates internally, output the best K for quality and diversity. For this session:

• Internal candidates generated: 5
• Selected for evaluation: 3 (best coverage of distinct mechanisms)
• Selection criteria:
  • Each hypothesis must cover a distinct causal mechanism (no overlapping explanations)
  • Each hypothesis must be falsifiable (explicit falsification condition stated upfront)
  • Each hypothesis must go beyond AR-00004 (no restatements of proxy clusters — new causal angles required)
  • Each hypothesis must be testable against existing Substrate data

Candidate Filtering

The 5 internal candidates covered: political agency, attention velocity, values fragmentation, economic precarity, and algorithmic curation. Economic precarity and algorithmic curation were filtered out:

• Economic precarity → too closely overlaps AR-00004's mental health proxy cluster; not a genuinely new causal angle
• Algorithmic curation → mechanistically a subset of H2 (attention velocity); insufficient independent variance

Final three selected: H1 (political), H2 (attentional), H3 (values-structural).

---

Phase 2: Hypothesis Evaluation — Science FullCycle

Pre-Commitment Protocol

Critical: Threshold locked before any evidence was examined. Pre-committed threshold: ≥3/5 predicted observations confirmed = Supported.

This prevents post-hoc threshold adjustment based on results. The threshold was fixed before examining any Substrate data for any of the three hypotheses.

Science FullCycle Steps (per hypothesis)

For each hypothesis:

1. State the hypothesis — causal mechanism, direction, and scope
2. State the falsification condition — what specific observation would definitively refute it
3. Derive 5 specific, independent predictions — each must be checkable against existing Substrate data
4. Check each prediction — confirmed (✅), disconfirmed (❌), or absent from data (⚠️ gap)
5. Apply pre-committed threshold — count ✅; ≥3 → Supported, <3 → Inconclusive, ≥1 ❌ → Refuted
6. Record verdict and data gaps

Evidence Standards

• Evidence must come from named Substrate datasets (no general knowledge claims)
• "Not in Substrate" counts as a data gap (⚠️), not a confirmation or refutation
• A gap does not downgrade Supported but limits confidence
• Contradictory evidence (❌) carries more weight than gaps (⚠️)

Verdict Taxonomy

Verdict	Criterion
✅ Supported	≥3/5 predictions confirmed, 0 refuted
⚠️ Inconclusive	<3/5 confirmed (data gaps or weak association, not refuted)
❌ Refuted	≥1 prediction directly contradicted by Substrate data

---

Quality Considerations

Strengths:

• Pre-commitment prevents Researcher Degrees of Freedom inflation
• All evidence from a single, auditable source (Substrate datasets)
• H3's key evidence is within-dataset (same WVS respondents, same waves) — strongest possible design given available data

Limitations:

• Substrate datasets are cross-sectional or aggregate; temporal ordering cannot be established without longitudinal individual-level data (SOEP)
• H3 finding is correlational — postmaterialism rising while satisfaction falling is consistent with PM causing the decline, but also with reverse causation or shared confounders (economic shocks, COVID-19 period)
• H1 and H2 rely on dataset-level associations across different surveys — ecological fallacy risk
• Cross-national comparison needed to rule out Germany-specific confounders for H3

Reproducibility: This protocol is fully reproducible given the same Substrate datasets. Any researcher with access to DE-World-Values, DE-Mental-Health, DE-Church-Exits, DE-Social-Isolation, DE-Platform-Media, and DE-Epistemic-Competence can re-run the evidence checks and verify the verdicts.