svemagie/Substrate
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
main
Substrate/Data/sources/WELLBEING_DATA_SOURCES.md

15 KiB
Raw Permalink Blame History

Wellbeing Data Sources - Implementation Guide

Created: 2025-10-27 Purpose: Document the five new wellbeing data sources added to Substrate to measure actual state of people

Overview

This document describes five critical data sources added to Substrate on 2025-10-27 to track human wellbeing beyond traditional economic indicators. These sources were selected based on:

  1. Free access with excellent APIs
  2. High quality and authoritative
  3. Leading indicators that reveal wellbeing before traditional metrics
  4. Behavioral truth - actions reveal reality surveys miss
  5. Coverage of critical dimensions - economic, health, social, environmental

The Five New Data Sources

DS-00004 — FRED Economic Wellbeing

Organization: Federal Reserve Bank of St. Louis API: https://api.stlouisfed.org/fred/ Update Frequency: Weekly to Annual (varies by indicator) Geographic Coverage: US National

Critical Indicators:

  • TDSP - Household Debt Service Ratio (quarterly) - Aggregate financial stress
  • DRCCLACBS - Credit Card Delinquency Rate (quarterly) - Consumer distress signal
  • STLFSI4 - Financial Stress Index (weekly!) - Real-time system stress
  • LNS13327709 - U-6 Underemployment Rate (monthly) - True labor slack
  • UEMP27OV - Long-term Unemployed 27+ weeks (monthly) - Structural problems
  • UMCSENT - Consumer Sentiment (monthly) - Economic confidence
  • SIPOVGINIUSA - GINI Index (annual) - Income inequality
  • MORTGAGE30US - 30-Year Mortgage Rate (weekly) - Housing affordability
  • MSPUS - Median Home Sales Price (quarterly) - Home price affordability
  • PSAVERT - Personal Saving Rate (monthly) - Financial resilience

Why It Matters:

  • Economic security is foundation for all wellbeing
  • Debt service ratio >12% indicates stress, >14% crisis
  • Financial stress index captures system-wide conditions
  • Free and comprehensive - best economic data available

Setup:

# Get free API key: https://fred.stlouisfed.org/docs/api/api_key.html
export FRED_API_KEY="your_key_here"
cd Data-Sources/DS-00004—FRED_Economic_Wellbeing
./update.ts

DS-00005 — CDC WONDER Mortality Database

Organization: Centers for Disease Control and Prevention (CDC) API: https://wonder.cdc.gov/controller/datarequest/ (XML) Update Frequency: Annual (with 1-2 year lag) Geographic Coverage: US National, State, County

Critical Indicators:

  • Drug Overdose Deaths (ICD-10: X40-X44, X60-X64, X85, Y10-Y14)
  • Opioid-Specific Deaths (T40.0-T40.4, T40.6)
  • Suicide Deaths (X60-X84, Y87.0, U03)
  • All-Cause Mortality Rates

Why It Matters:

  • Leading indicators - Overdoses and suicides precede economic decline
  • Behavioral truth - Deaths reveal desperation surveys miss
  • County-level granularity - Shows which communities are suffering
  • "Deaths of despair" - Captures breakdown in social fabric and hope
  • Only official source for county-level crisis mortality

Unique Insight:

  • These are not random health events - they're signals of community breakdown
  • Geographic patterns show "left behind" populations
  • Crisis indicators that traditional wellbeing metrics miss entirely

Setup:

cd Data-Sources/DS-00005—CDC_WONDER_Mortality
./update.ts
# No API key required - public access

DS-00006 — Census ACS Social Wellbeing

Organization: US Census Bureau API: https://api.census.gov/data/{year}/acs/acs1 Update Frequency: Annual (1-year and 5-year estimates) Geographic Coverage: National, State, County, City, Census Tract

Critical Indicators:

  • B11001_008E - 1-Person Households (living alone) - Social isolation
  • B08303_001E - Mean Travel Time to Work - Time poverty
  • B08303_013E - Commute 60+ minutes - Extreme time poverty
  • B28002_013E - No Internet Access at Home - Digital divide
  • B19013_001E - Median Household Income - Economic security
  • B25064_001E - Median Gross Rent - Housing affordability
  • B23025_005E - Unemployed Population - Labor market health

Why It Matters:

  • Social connection - Living alone rates reveal structural isolation
  • Time poverty - Long commutes reduce social connection, increase stress
  • Digital divide - Internet access = opportunity access in modern economy
  • Most granular source - Down to census tract level (neighborhood data)
  • Denominators - Population data needed to calculate rates

Unique Insight:

  • You can be economically comfortable but socially isolated (suburban paradox)
  • Time poverty (commute) often invisible in income statistics
  • Structural determinants you can't "self-care" your way out of

Setup:

# Get free API key: https://api.census.gov/data/key_signup.html
export CENSUS_API_KEY="your_key_here"
cd Data-Sources/DS-00006—Census_ACS_Social_Wellbeing
./update.ts

DS-00007 — BLS JOLTS Labor Market

Organization: Bureau of Labor Statistics (BLS) API: https://api.bls.gov/publicAPI/v2/timeseries/data/ Update Frequency: Monthly (with ~6 week lag) Geographic Coverage: US National, some State

Critical Indicators (via FRED for reliability):

  • JTSQUR - Quit Rate (Total Nonfarm) - MOST IMPORTANT
  • JTSJOR - Job Openings Rate - Opportunity availability
  • JTSHIR - Hire Rate - Labor market dynamism
  • JTSLD - Layoff and Discharge Rate - Involuntary separations
  • JTSTSR - Total Separations Rate - Overall turnover

Why It Matters - The "Permission to Quit Index":

  • People only quit when they have options - Quit rate measures worker agency
  • High quit rate = Worker empowerment, confidence, economic security
  • Low quit rate during "good economy" = Trapped workers (hidden desperation)
  • Leading indicator of wage growth (quits force employers to raise wages)
  • Reveals worker experience that GDP and unemployment miss

Unique Framework:

  • "Permission to Quit" measures economic freedom and worker dignity
  • Distinguishes voluntary (quits) from involuntary (layoffs) separations
  • Worker-centric view of economy (not just employer/investor perspective)

Setup:

# Optional: Get free BLS API key for higher rate limits
# https://www.bls.gov/developers/home.htm
export BLS_API_KEY="your_key_here"  # Optional
export FRED_API_KEY="your_key_here"  # Required (data via FRED)
cd Data-Sources/DS-00007—BLS_JOLTS_Labor_Market
./update.ts

Note: Update script uses FRED API to access JOLTS data (more reliable than direct BLS API). Original BLS series IDs changed format in 2020.

DS-00008 — EPA Air Quality System

Organization: Environmental Protection Agency (EPA) API: https://aqs.epa.gov/data/api/ Update Frequency: Hourly (real-time) to Annual summaries Geographic Coverage: US National, State, County, Monitoring Station

Critical Indicators:

  • 88101 - PM2.5 (fine particulate matter) - MOST CRITICAL
  • 44201 - Ozone (O3) - Respiratory and cardiovascular impacts
  • 42401 - Sulfur Dioxide (SO2)
  • 42101 - Carbon Monoxide (CO)
  • 42602 - Nitrogen Dioxide (NO2)
  • 81102 - PM10 (coarse particulate matter)

Why It Matters - Environmental Justice:

  • You cannot "self-care" your way out of breathing toxic air
  • PM2.5 reduces life expectancy by months to years
  • Environmental injustice - Low-income communities disproportionately exposed
  • Structural determinant - ZIP code determines air quality, not personal choice
  • Measurable, actionable, preventable health risk

Health Impacts:

  • PM2.5: Mortality, cardiovascular disease, respiratory disease, cognitive decline
  • Ozone: Respiratory inflammation, asthma exacerbation
  • Long-term exposure in top decile can reduce life expectancy 1-3 years

Unique Insight:

  • Air quality is a structural wellbeing constraint like poverty
  • Policy visibility through monitoring (gaps in underserved areas = "data invisibility")
  • Environmental health reveals that wellbeing requires collective action, not just individual choices

Setup:

# Register for free API key: aqs.support@epa.gov
export EPA_AQS_EMAIL="your_email@example.com"
export EPA_AQS_KEY="your_key_here"
cd Data-Sources/DS-00008—EPA_Air_Quality_System
./update.ts --year 2023 --states CA,NY,TX

Integrated Wellbeing Framework

These five sources cover the critical dimensions of human wellbeing:

1. Economic Security (FRED)

  • Financial stress and debt burden
  • Employment quality (not just quantity)
  • Housing affordability
  • Income inequality

2. Health & Crisis (CDC WONDER)

  • Deaths of despair (overdoses, suicides)
  • All-cause mortality trends
  • Community-level health breakdown
  • Leading indicators of social collapse

3. Social Connection (Census ACS)

  • Structural isolation (living alone)
  • Time poverty (commute duration)
  • Digital divide (internet access)
  • Neighborhood characteristics

4. Work & Purpose (BLS JOLTS)

  • Worker agency (quit rate)
  • Economic opportunity (job openings)
  • Labor market dynamism
  • Voluntary vs involuntary separation

5. Environmental Health (EPA AQS)

  • Air quality and life expectancy
  • Environmental justice
  • Structural health determinants
  • Geographic inequality

Composite Wellbeing Indices

Based on the research, consider creating these composite indices:

Financial Stress Composite (FSC)

FSC = weighted_average([
  TDSP (debt service ratio),
  DRCCLACBS (credit card delinquency),
  Eviction rates (external source),
  STLFSI4 (financial stress index)
])

Alert Thresholds: >50 = elevated stress, >70 = crisis

Crisis Alert Composite (CAC)

CAC = normalized_sum([
  Drug overdose deaths (CDC WONDER),
  Suicide rates (CDC WONDER),
  Long-term unemployment (FRED)
])

Leading indicator - Spikes before economic metrics decline

Community Health Composite (CHC)

CHC = inverse_weighted_average([
  Living alone rate (Census ACS),
  Long commute rate (Census ACS),
  No internet access (Census ACS)
])

Measures social infrastructure - Connection and opportunity access

Worker Agency Index (WAI)

WAI = weighted_average([
  Quit rate (BLS JOLTS),
  Job openings rate (BLS JOLTS),
  Inverse of long-term unemployment (FRED)
])

"Permission to Quit" - Economic freedom and worker dignity

Environmental Health Index (EHI)

EHI = inverse_weighted_average([
  PM2.5 concentration (EPA AQS),
  Ozone concentration (EPA AQS),
  Days exceeding AQI 100
])

Structural health determinant - Collective wellbeing constraint

Update Schedule Recommendations

Weekly:

  • FRED indicators (captures high-frequency economic stress)
  • EPA AQS (tracks air quality events)

Monthly:

  • FRED monthly indicators (unemployment, sentiment, saving rate)
  • BLS JOLTS (labor market health)

Quarterly:

  • FRED quarterly indicators (debt service, home prices)

Annual:

  • Census ACS (social wellbeing indicators)
  • CDC WONDER (mortality data has 1-2 year lag anyway)

Data Quality Notes

Completeness

  • FRED: Excellent (long time series, rarely missing data)
  • CDC WONDER: Good (cell suppression for privacy in low-count cells)
  • Census ACS: Excellent (comprehensive US coverage)
  • BLS JOLTS: Good (national reliable, state-level variable)
  • EPA AQS: Good (monitoring gaps in rural areas and some underserved communities)

Timeliness

  • FRED: 1 week to 3 months depending on indicator
  • CDC WONDER: 1-2 year lag (deaths require coding)
  • Census ACS: 6-12 months (annual release)
  • BLS JOLTS: 6 weeks (faster than most labor data)
  • EPA AQS: Real-time to 6 months

Geographic Granularity

  • FRED: National only for wellbeing indicators (some state data available)
  • CDC WONDER: National, State, County (excellent)
  • Census ACS: National, State, County, City, Census Tract (exceptional)
  • BLS JOLTS: National, limited State (national most reliable)
  • EPA AQS: Monitoring station (lat/long), aggregates to county/state

Known Limitations

What These Sources CANNOT Tell You

  1. Individual-level wellbeing - All are aggregated data (use surveys for individual experience)
  2. Real-time wellbeing - All have lag (1 week to 2 years)
  3. Causation - Correlation only (use experimental designs for causation)
  4. Subjective experience - Behavioral/objective only (use Gallup/Pew for perceptions)
  5. International comparison - US-only (use WHO GHO, UN SDG for global)

Gaps to Fill with Additional Sources

  • Food insecurity - USDA ERS needed
  • Homelessness - HUD Point-in-Time Count needed
  • Substance abuse treatment - SAMHSA needed
  • Mental health service utilization - Multiple sources needed
  • Sleep quality - CDC NHIS or NSF needed
  • Volunteering/civic engagement - AmeriCorps/Pew needed

Philosophy: Knowing the Actual State of People

Why this matters:

Traditional wellbeing measurement focuses on:

  • GDP growth (economic output, not wellbeing)
  • Unemployment rate (misses underemployment, quality)
  • Survey happiness (subject to response bias, optimism)

These new sources focus on:

  • Crisis indicators (overdoses, suicides) - Reveal breakdown
  • Behavioral truth (quit rates, debt delinquency) - Actions > words
  • Structural determinants (air quality, commute times) - Constraints on flourishing
  • Leading indicators (financial stress before recession) - Early warning
  • Geographic granularity (county-level) - No one left invisible

Core insight:

"If we measure only GDP and unemployment, we will miss the slow-motion collapse of human thriving happening in plain sight."

Purpose:

"When we theorize or propose solutions, we are informed by the actual state of people - not abstractions, not averages, not GDP."

Next Steps

  1. Test all update scripts with valid API keys
  2. Run initial data fetches to populate data directories
  3. Create composite indices (FSC, CAC, CHC, WAI, EHI)
  4. Build dashboards for visualization
  5. Establish alert thresholds for crisis detection
  6. Cross-reference with Substrate Problems and Solutions
  7. Add remaining sources from research (food insecurity, homelessness, etc.)
  8. Geographic analysis - County-level maps of wellbeing
  9. Time-series analysis - Trend detection and forecasting
  10. Integration - Combine sources to find feedback loops and cascading failures

Credits

Research Date: 2025-10-27 Researcher: Kai (Claude Code) Research Scope: 100+ datasets evaluated, 5 prioritized for implementation Selection Criteria: Free access, excellent APIs, high quality, leading indicators, behavioral truth Implementation: Complete substrate-style documentation for each source

Research Documents:

  • /Users/daniel/.claude/history/research/2025-10/2025-10-27_wellbeing-substrate-datasets/
  • FRED research: 50+ series IDs identified
  • Pew/Gallup research: 15 major datasets cataloged
  • Alternative sources: 37 indicators across 6 categories

END OF DOCUMENT

Reference in New Issue View Git Blame Copy Permalink