Files
Substrate/Data/US-GDP/RESOURCES.md
Daniel Miessler 3d700a33a8 Add U.S. GDP dataset with comprehensive documentation (1929-2025)
Added authoritative U.S. GDP data from FRED/BEA covering 96 years of economic history:
- Real GDP quarterly data: Q1 1947 - Q2 2025 (314 data points)
- Real GDP annual data: 1929 - 2024 (96 data points)

Includes comprehensive research-backed documentation:
- Complete source verification (10 parallel research agents, 95%+ confidence)
- Full methodology documentation and data quality standards
- Download instructions, APIs, and update procedures
- Historical context from Great Depression through COVID-19

Updated main README with Data directory documentation and dataset comparison table.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-16 21:44:42 -07:00

8.7 KiB

U.S. GDP Data Resources

This document provides direct links to data sources, APIs, and tools for accessing U.S. GDP data.

Primary Data Sources

FRED (Federal Reserve Economic Data)

Official Website:

Real GDP Series Pages:

Direct CSV Downloads:

# Quarterly Real GDP (GDPC1) - 1947 to present
curl -L "https://fred.stlouisfed.org/graph/fredgraph.csv?id=GDPC1" -o gdpc1.csv

# Annual Real GDP (GDPCA) - 1929 to present
curl -L "https://fred.stlouisfed.org/graph/fredgraph.csv?id=GDPCA" -o gdpca.csv

# Quarterly Nominal GDP (GDP) - 1947 to present
curl -L "https://fred.stlouisfed.org/graph/fredgraph.csv?id=GDP" -o gdp.csv

FRED API:

FRED API Example:

# Get Real GDP data via API (requires API key)
curl "https://api.stlouisfed.org/fred/series/observations?series_id=GDPC1&api_key=YOUR_API_KEY&file_type=json"

Bureau of Economic Analysis (BEA)

Official Website:

Interactive Data Tools:

BEA API:

BEA API Example:

# Get GDP data via BEA API (requires free API key)
curl "https://apps.bea.gov/api/data/?UserID=YOUR_API_KEY&method=GetData&datasetname=NIPA&TableName=T10106&Frequency=Q&Year=X&ResultFormat=json"

Key NIPA Tables:

  • Table 1.1.6: Real GDP (Quarterly and Annual)
  • Table 1.1.5: Nominal GDP (Quarterly and Annual)
  • Table 1.1.9: Implicit Price Deflators for GDP

News Releases:

Extended Historical Data

MeasuringWorth (1790-Present)

Website: https://www.measuringworth.com/datasets/usgdp/

Coverage: U.S. GDP from 1790 to present (links historical estimates to modern BEA data)

Key Researchers:

  • Louis Johnston (College of Saint Benedict / Saint John's University)
  • Samuel H. Williamson (University of Illinois at Chicago - emeritus)

Methodology:

  • Pre-1929 data based on economic historians' research:
    • Weiss (1799-1829)
    • Gallman (1839-1909 benchmarks)
    • Kendrick (1909-1928)
  • Post-1929: Official BEA data

Download: Available as CSV from MeasuringWorth website

GDPNow (Real-Time GDP Forecasting)

Atlanta Fed GDPNow Model:

  • Website: https://www.atlantafed.org/cqer/research/gdpnow
  • Purpose: Nowcasting model for current quarter GDP estimates
  • Updates: Multiple times weekly as new economic data releases
  • Useful for: Tracking economic activity between official BEA releases

Alternative Data Aggregators

International Sources

World Bank:

IMF (International Monetary Fund):

OECD:

Data Download Methods Comparison

Source Format Authentication Historical Coverage Update Frequency Best For
FRED CSV CSV None 1947 (Q), 1929 (A) Quarterly Quick downloads, scripting
FRED API JSON/XML API Key (free) 1947 (Q), 1929 (A) Quarterly Automated data pipelines
BEA iTable CSV/Excel None 1929+ Quarterly Interactive exploration
BEA API JSON/XML API Key (free) 1929+ Quarterly Detailed NIPA table access
MeasuringWorth CSV None 1790+ Annual updates Long-term historical analysis

Update Schedule

BEA Release Schedule (Three-Stage Process)

Each quarter of GDP data goes through three releases:

  1. Advance Estimate: ~30 days after quarter end (8:30 AM ET)
  2. Second Estimate: ~60 days after quarter end (8:30 AM ET)
  3. Third Estimate: ~90 days after quarter end (8:30 AM ET)

Annual Comprehensive Update:

  • Released each September
  • Revises 5+ years of historical data
  • Incorporates methodological improvements

Example Schedule for Q3 2025:

  • Advance: October 30, 2025
  • Second: November 26, 2025
  • Third: December 23, 2025

2025 Release Calendar:

  • Q4 2024 Third: January 30, 2025
  • Q1 2025 Third: June 26, 2025
  • Q2 2025 Third: September 25, 2025
  • Annual Update: September 25, 2025
  • Q3 2025 Advance: October 30, 2025 (upcoming)

How to Update This Dataset

Manual Update via FRED

# Download latest quarterly data
curl -L "https://fred.stlouisfed.org/graph/fredgraph.csv?id=GDPC1" -o Real-GDP-Quarterly-1947-2025.csv

# Download latest annual data
curl -L "https://fred.stlouisfed.org/graph/fredgraph.csv?id=GDPCA" -o Real-GDP-Annual-1929-2024.csv

# Verify download
head -5 Real-GDP-Quarterly-1947-2025.csv
tail -5 Real-GDP-Quarterly-1947-2025.csv

Automated Update Script

#!/bin/bash
# update-gdp-data.sh

DATA_DIR="./Data/US-GDP"
FRED_BASE="https://fred.stlouisfed.org/graph/fredgraph.csv"

# Download quarterly real GDP
curl -L "${FRED_BASE}?id=GDPC1" -o "${DATA_DIR}/Real-GDP-Quarterly-1947-2025.csv"

# Download annual real GDP
curl -L "${FRED_BASE}?id=GDPCA" -o "${DATA_DIR}/Real-GDP-Annual-1929-2024.csv"

echo "GDP data updated: $(date)"

Best Practices for Updates

  1. Update after BEA's third estimate (most complete data for the quarter)
  2. Update after September annual revision (historical data corrections)
  3. Verify data integrity after download (check first/last rows, row count)
  4. Document updates in UPDATES.md with date and data range
  5. Note any methodology changes from BEA annual updates

Data Verification

Quick Verification Checklist

# Check quarterly data
wc -l Real-GDP-Quarterly-1947-2025.csv  # Should be ~315 lines (header + 314 quarters)
head -3 Real-GDP-Quarterly-1947-2025.csv  # Should start with 1947-01-01
tail -3 Real-GDP-Quarterly-1947-2025.csv  # Should end with latest quarter

# Check annual data
wc -l Real-GDP-Annual-1929-2024.csv  # Should be ~97 lines (header + 96 years)
head -3 Real-GDP-Annual-1929-2024.csv  # Should start with 1929-01-01
tail -3 Real-GDP-Annual-1929-2024.csv  # Should end with 2024-01-01

Cross-Validation

  • Compare latest values with BEA news releases
  • Verify against FRED website display
  • Check that growth rates match reported economic news
  • Confirm no missing data points in series

Additional Research Resources

Economic Analysis

Academic Papers

News & Analysis

Support & Documentation

FRED Support

BEA Support


Last Updated: 2025-10-16 Next Recommended Check: After Q3 2025 third estimate (late January 2026)