Substrate/QUICK_REFERENCE.md

Substrate Quick Reference

Command cheatsheet for working with Substrate.

---

Setup

# Clone repository
git clone https://github.com/danielmiessler/Substrate.git
cd Substrate

# Install Bun (if needed)
curl -fsSL https://bun.sh/install | bash

# Install dependencies
bun install

---

Browse Data

# List core datasets
ls Data/

# List wellbeing data sources
ls Data-Sources/

# View a specific dataset
cat Data/US-GDP/us-gdp-annual.csv | head -20

# Open in spreadsheet app (macOS)
open Data/US-GDP/us-gdp-annual.csv

---

Update Data

Single Dataset

cd Data/US-GDP
bun run update.ts

Wellbeing Sources (Require API Keys)

# Set API key first
export FRED_API_KEY="your_key"

# Then run update
cd Data-Sources/DS-00004—FRED_Economic_Wellbeing
bun run update.ts

All Datasets

bun run scripts/update-all.ts

---

API Keys

Source	Get Key	Env Variable
FRED	fred.stlouisfed.org/docs/api	FRED_API_KEY
Census	api.census.gov/data/key_signup	CENSUS_API_KEY
EPA	Email: aqs.support@epa.gov	EPA_KEY
BLS	bls.gov/developers/home	BLS_API_KEY
CDC WONDER	No key needed	—

# Set all keys at once
export FRED_API_KEY="xxx"
export CENSUS_API_KEY="xxx"
export EPA_KEY="xxx"
export BLS_API_KEY="xxx"

---

Contributing

Fork & Clone

# Fork on GitHub, then:
git clone https://github.com/YOUR_USERNAME/Substrate.git
cd Substrate
git remote add upstream https://github.com/danielmiessler/Substrate.git

Create Branch

git checkout -b add-my-contribution

Commit & Push

git add .
git commit -m "Add: description"
git push origin add-my-contribution

Stay Updated

git fetch upstream
git merge upstream/main

---

Directory Quick Reference

Directory	Contains	ID Format
Data/	Core datasets (US, DE, EU)	Folder names
Data/sources/	Source catalog	DS-00001 – DS-00020
Plans/	Country plans (US, DE)	PL-00001
Problems/	Documented challenges	PR-00001
Solutions/	Proven approaches	SO-00001
Arguments/	Reasoning chains	AR-00001
Claims/	Evidence-linked assertions	CL-00001
Ideas/	Frameworks	ID-00001
People/	Researchers	PE-00001
Organizations/	Groups	OR-00001
Projects/	Initiatives	PJ-00001
Values/	Principles	VA-00001

---

Data Sources Quick Reference

🌍 Global

ID	Source	Key Indicators
DS-00001	WHO Global Health	Health indicators (194 countries)
DS-00002	UN SDG	Sustainable Development Goals
DS-00003	World Bank	Development metrics

🇺🇸 United States

ID	Source	Key Indicators
DS-00004	FRED Economic	Debt, unemployment, inequality
DS-00005	CDC WONDER	Overdoses, suicides, mortality
DS-00006	Census ACS	Social isolation, commute, digital divide
DS-00007	BLS JOLTS	Quit rate, job openings, layoffs
DS-00008	EPA Air Quality	PM2.5, ozone, air quality
DS-00009	EIA Energy	Energy production/consumption
DS-00010	Treasury Fiscal	Federal revenue/spending

🇩🇪 Germany

ID	Source	Key Indicators
DS-00011	Lobbyregister Bundestag	Registered lobbyists, expenditure
DS-00012	SMARD Strommarkt	Energy mix, renewable share
DS-00013	Bundeshaushalt	Federal budget allocation
DS-00014	DIP Bundestag	Legislative activity, Drucksachen
DS-00015	Platform Media	Platform usage, news consumption
DS-00016	Epistemic Competence	Media literacy, PISA scores
DS-00017	Social Mobility	Gymnasium rates, education spending
DS-00018	RKI AMELAG Wastewater	SARS-CoV-2 wastewater levels
DS-00019	Wealth Distribution	Wealth Gini, top shares, inheritance
DS-00020	EU Wealth Inequality	Cross-country Gini, DWA shares
DS-00021	Church Exits (EKD/DBK)	Kirchenaustritte, membership
DS-00022	Common Metrics	29 economic/demographic indicators
DS-00023	Democracy Metrics	V-Dem, press freedom, turnout
DS-00024	Mental Health	Engagement, suicide, sick days
DS-00025	Social Isolation	Single households, loneliness
DS-00026	Wellbeing	Life satisfaction, meaning (Eurostat)
DS-00027	World Values	Inglehart dimensions (WVS/EVS)

---

Common Tasks

Find a Dataset

# Search by name
ls Data/ | grep -i gdp
ls Data-Sources/ | grep -i fred

# Search in README files
grep -r "unemployment" Data-Sources/*/README.md

Check Data Freshness

# View last update time
ls -la Data/US-GDP/*.csv
cat Data/US-GDP/update-log.md | tail -20

Validate Data

# Count rows
wc -l Data/US-GDP/us-gdp-annual.csv

# Check for missing values
grep -c ",," Data/US-GDP/us-gdp-annual.csv

---

Links

• README: ./README.md
• Getting Started: ./GETTING_STARTED.md
• Updates: ./UPDATES.md
• Data Philosophy: ./Data/README.md
• Library Science: ./Data/README-LIBRARY-SCIENCE.md

---

← Back to README | Getting Started →