# Substrate Quick Reference

Command cheatsheet for working with Substrate.

---

## Setup

```bash
# Clone repository
git clone https://github.com/danielmiessler/Substrate.git
cd Substrate

# Install Bun (if needed)
curl -fsSL https://bun.sh/install | bash

# Install dependencies
bun install
```

---

## Browse Data

```bash
# List core datasets
ls Data/

# List wellbeing data sources
ls Data-Sources/

# View a specific dataset
cat Data/US-GDP/us-gdp-annual.csv | head -20

# Open in spreadsheet app (macOS)
open Data/US-GDP/us-gdp-annual.csv
```

---

## Update Data

### Single Dataset

```bash
cd Data/US-GDP
bun run update.ts
```

### Wellbeing Sources (Require API Keys)

```bash
# Set API key first
export FRED_API_KEY="your_key"

# Then run update
cd Data-Sources/DS-00004—FRED_Economic_Wellbeing
bun run update.ts
```

### All Datasets

```bash
bun run scripts/update-all.ts
```

---

## API Keys

| Source | Get Key | Env Variable |
|--------|---------|--------------|
| FRED | [fred.stlouisfed.org/docs/api](https://fred.stlouisfed.org/docs/api/api_key.html) | `FRED_API_KEY` |
| Census | [api.census.gov/data/key_signup](https://api.census.gov/data/key_signup.html) | `CENSUS_API_KEY` |
| EPA | Email: aqs.support@epa.gov | `EPA_KEY` |
| BLS | [bls.gov/developers/home](https://www.bls.gov/developers/home.htm) | `BLS_API_KEY` |
| CDC WONDER | No key needed | — |

```bash
# Set all keys at once
export FRED_API_KEY="xxx"
export CENSUS_API_KEY="xxx"
export EPA_KEY="xxx"
export BLS_API_KEY="xxx"
```

---

## Contributing

### Fork & Clone

```bash
# Fork on GitHub, then:
git clone https://github.com/YOUR_USERNAME/Substrate.git
cd Substrate
git remote add upstream https://github.com/danielmiessler/Substrate.git
```

### Create Branch

```bash
git checkout -b add-my-contribution
```

### Commit & Push

```bash
git add .
git commit -m "Add: description"
git push origin add-my-contribution
```

### Stay Updated

```bash
git fetch upstream
git merge upstream/main
```

---

## Directory Quick Reference

| Directory | Contains | ID Format |
|-----------|----------|-----------|
| `Data/` | Core datasets (US, DE, EU) | Folder names |
| `Data/sources/` | Source catalog | `DS-00001` – `DS-00020` |
| `Plans/` | Country plans (US, DE) | `PL-00001` |
| `Problems/` | Documented challenges | `PR-00001` |
| `Solutions/` | Proven approaches | `SO-00001` |
| `Arguments/` | Reasoning chains | `AR-00001` |
| `Claims/` | Evidence-linked assertions | `CL-00001` |
| `Ideas/` | Frameworks | `ID-00001` |
| `People/` | Researchers | `PE-00001` |
| `Organizations/` | Groups | `OR-00001` |
| `Projects/` | Initiatives | `PJ-00001` |
| `Values/` | Principles | `VA-00001` |

---

## Data Sources Quick Reference

**🌍 Global**

| ID | Source | Key Indicators |
|----|--------|----------------|
| DS-00001 | WHO Global Health | Health indicators (194 countries) |
| DS-00002 | UN SDG | Sustainable Development Goals |
| DS-00003 | World Bank | Development metrics |

**🇺🇸 United States**

| ID | Source | Key Indicators |
|----|--------|----------------|
| DS-00004 | FRED Economic | Debt, unemployment, inequality |
| DS-00005 | CDC WONDER | Overdoses, suicides, mortality |
| DS-00006 | Census ACS | Social isolation, commute, digital divide |
| DS-00007 | BLS JOLTS | Quit rate, job openings, layoffs |
| DS-00008 | EPA Air Quality | PM2.5, ozone, air quality |
| DS-00009 | EIA Energy | Energy production/consumption |
| DS-00010 | Treasury Fiscal | Federal revenue/spending |

**🇩🇪 Germany**

| ID | Source | Key Indicators |
|----|--------|----------------|
| DS-00011 | Lobbyregister Bundestag | Registered lobbyists, expenditure |
| DS-00012 | SMARD Strommarkt | Energy mix, renewable share |
| DS-00013 | Bundeshaushalt | Federal budget allocation |
| DS-00014 | DIP Bundestag | Legislative activity, Drucksachen |
| DS-00015 | Platform Media | Platform usage, news consumption |
| DS-00016 | Epistemic Competence | Media literacy, PISA scores |
| DS-00017 | Social Mobility | Gymnasium rates, education spending |
| DS-00018 | RKI AMELAG Wastewater | SARS-CoV-2 wastewater levels |
| DS-00019 | Wealth Distribution | Wealth Gini, top shares, inheritance |
| DS-00020 | EU Wealth Inequality | Cross-country Gini, DWA shares |
| DS-00021 | Church Exits (EKD/DBK) | Kirchenaustritte, membership |
| DS-00022 | Common Metrics | 29 economic/demographic indicators |
| DS-00023 | Democracy Metrics | V-Dem, press freedom, turnout |
| DS-00024 | Mental Health | Engagement, suicide, sick days |
| DS-00025 | Social Isolation | Single households, loneliness |
| DS-00026 | Wellbeing | Life satisfaction, meaning (Eurostat) |
| DS-00027 | World Values | Inglehart dimensions (WVS/EVS) |

---

## Common Tasks

### Find a Dataset

```bash
# Search by name
ls Data/ | grep -i gdp
ls Data-Sources/ | grep -i fred

# Search in README files
grep -r "unemployment" Data-Sources/*/README.md
```

### Check Data Freshness

```bash
# View last update time
ls -la Data/US-GDP/*.csv
cat Data/US-GDP/update-log.md | tail -20
```

### Validate Data

```bash
# Count rows
wc -l Data/US-GDP/us-gdp-annual.csv

# Check for missing values
grep -c ",," Data/US-GDP/us-gdp-annual.csv
```

---

## Links

- **README**: [./README.md](./README.md)
- **Getting Started**: [./GETTING_STARTED.md](./GETTING_STARTED.md)
- **Updates**: [./UPDATES.md](./UPDATES.md)
- **Data Philosophy**: [./Data/README.md](./Data/README.md)
- **Library Science**: [./Data/README-LIBRARY-SCIENCE.md](./Data/README-LIBRARY-SCIENCE.md)

---

**[← Back to README](./README.md)** | **[Getting Started →](./GETTING_STARTED.md)**