Substrate/research/meaning-crisis-hypotheses-2026-04/METHODOLOGY.md

# Methodology

**Research Project:** Meaning Crisis — Causal Hypotheses (PR-00001)
**Date:** 2026-04-22

---

## Research Design

Two-phase process: (1) hypothesis generation via BeCreative, (2) hypothesis evaluation via Science FullCycle protocol.

**Research Duration:** Single session, 2026-04-22
**Substrate Datasets Consulted:** 6 (DE-World-Values, DE-Mental-Health, DE-Church-Exits, DE-Social-Isolation, DE-Platform-Media, DE-Epistemic-Competence)
**External sources queried:** None — evidence exclusively from curated Substrate datasets

---

## Phase 1: Hypothesis Generation — BeCreative (Verbalized Sampling)

### Protocol

BeCreative uses Verbalized Sampling: generate N candidates internally, output the best K for quality and diversity. For this session:

- **Internal candidates generated:** 5
- **Selected for evaluation:** 3 (best coverage of distinct mechanisms)
- **Selection criteria:**
  - Each hypothesis must cover a distinct causal mechanism (no overlapping explanations)
  - Each hypothesis must be falsifiable (explicit falsification condition stated upfront)
  - Each hypothesis must go beyond AR-00004 (no restatements of proxy clusters — new causal angles required)
  - Each hypothesis must be testable against existing Substrate data

### Candidate Filtering

The 5 internal candidates covered: political agency, attention velocity, values fragmentation, economic precarity, and algorithmic curation. Economic precarity and algorithmic curation were filtered out:
- **Economic precarity** → too closely overlaps AR-00004's mental health proxy cluster; not a genuinely new causal angle
- **Algorithmic curation** → mechanistically a subset of H2 (attention velocity); insufficient independent variance

Final three selected: H1 (political), H2 (attentional), H3 (values-structural).

---

## Phase 2: Hypothesis Evaluation — Science FullCycle

### Pre-Commitment Protocol

**Critical:** Threshold locked before any evidence was examined. Pre-committed threshold: **≥3/5 predicted observations confirmed = Supported**.

This prevents post-hoc threshold adjustment based on results. The threshold was fixed before examining any Substrate data for any of the three hypotheses.

### Science FullCycle Steps (per hypothesis)

For each hypothesis:

1. **State the hypothesis** — causal mechanism, direction, and scope
2. **State the falsification condition** — what specific observation would definitively refute it
3. **Derive 5 specific, independent predictions** — each must be checkable against existing Substrate data
4. **Check each prediction** — confirmed (✅), disconfirmed (❌), or absent from data (⚠️ gap)
5. **Apply pre-committed threshold** — count ✅; ≥3 → Supported, <3 → Inconclusive, ≥1 ❌ → Refuted
6. **Record verdict and data gaps**

### Evidence Standards

- Evidence must come from named Substrate datasets (no general knowledge claims)
- "Not in Substrate" counts as a data gap (⚠️), not a confirmation or refutation
- A gap does not downgrade Supported but limits confidence
- Contradictory evidence (❌) carries more weight than gaps (⚠️)

### Verdict Taxonomy

| Verdict | Criterion |
|---|---|
| ✅ Supported | ≥3/5 predictions confirmed, 0 refuted |
| ⚠️ Inconclusive | <3/5 confirmed (data gaps or weak association, not refuted) |
| ❌ Refuted | ≥1 prediction directly contradicted by Substrate data |

---

## Quality Considerations

**Strengths:**
- Pre-commitment prevents Researcher Degrees of Freedom inflation
- All evidence from a single, auditable source (Substrate datasets)
- H3's key evidence is within-dataset (same WVS respondents, same waves) — strongest possible design given available data

**Limitations:**
- Substrate datasets are cross-sectional or aggregate; temporal ordering cannot be established without longitudinal individual-level data (SOEP)
- H3 finding is correlational — postmaterialism rising while satisfaction falling is consistent with PM causing the decline, but also with reverse causation or shared confounders (economic shocks, COVID-19 period)
- H1 and H2 rely on dataset-level associations across different surveys — ecological fallacy risk
- Cross-national comparison needed to rule out Germany-specific confounders for H3

**Reproducibility:**
This protocol is fully reproducible given the same Substrate datasets. Any researcher with access to DE-World-Values, DE-Mental-Health, DE-Church-Exits, DE-Social-Isolation, DE-Platform-Media, and DE-Epistemic-Competence can re-run the evidence checks and verify the verdicts.