Substrate/Data/DE-Common-Metrics/source.md

# DE-Common-Metrics — Source Documentation

## Data Sources

### Destatis GENESIS (Statistisches Bundesamt)
- **URL:** https://www-genesis.destatis.de/genesisWS/rest/2020/data/table
- **Auth:** HTTP headers `username` + `password` (env: `DESTATIS_USERNAME`, `DESTATIS_PASSWORD`)
- **Method:** POST, body `application/x-www-form-urlencoded`
- **Body params:** `name=TABLE_CODE&area=all&compress=false&transpose=false&language=de`
- **Response:** JSON with `Object.Content` = semicolon-delimited CSV text
- **Rate limit:** ~1.5 req/s authenticated; skill uses 600ms delay
- **Metrics:** Industrial production, CPI, PPI, import prices, housing completions/permits, labor force, wages, demographics, health

### Bundesbank SDMX Web Service
- **URL:** https://api.statistiken.bundesbank.de/rest/data/{FLOW}/{KEY}?lastNObservations=1&format=sdmx_json
- **Auth:** None (public API)
- **Format:** SDMX-JSON; response wrapped under `data` key
- **Series ID format:** `FLOW/KEY` e.g. `BBIN1/M.D0.ECB.ECBREF.EUR.ME`
- **Rate limit:** No documented limit; skill uses 200ms delay
- **Metrics:** GDP, house prices, ECB rate, Bund yields, EUR/USD exchange rate, mortgage rates

### BA fuer Arbeit REST
- **URL:** https://rest.arbeitsagentur.de/infosysbub/
- **Auth:** OAuth2 client credentials (as of 2026-04-18)
- **Format:** JSON with `_embedded.statistiken` structure
- **Rate limit:** 1 req/s documented; skill uses 1100ms delay
- **Metrics:** Unemployment rate, employment rate, job openings, BA-X index, short-time workers
- **Status:** **BLOCKED — API now returns HTTP 403.** Requires OAuth2 client credentials token. No public alternative found yet. All 5 employment metrics currently failing (DE_UNRATE, DE_EMPRATE, DE_JOLTS, DE_BAX, DE_KURZARBEIT).

### ZEW Economic Sentiment
- **URL:** https://www.zew.de/en/publications/zew-indicator-of-economic-sentiment
- **Auth:** None (CSV download)
- **Format:** CSV
- **Rate limit:** Single download per run
- **Status:** Adapter currently failing — needs investigation

## Update Command

```bash
bun ~/.claude/skills/Metrics/Tools/UpdateMetrics.ts --country de
```

## Output Files

| File | Description |
|------|-------------|
| `DE-Common-Metrics.md` | Auto-generated dashboard with current values per category |
| `de-metrics-current.csv` | Latest value per metric (overwritten each run) |
| `de-metrics-historical.csv` | Appended on each run — full time series |

## Bundesbank Series ID Mapping

| Metric ID | Series ID | Dataflow | Description |
|-----------|-----------|----------|-------------|
| DE_GDP_NOM | `BBNZ1/Q.DE.N.G.0000.A` | National Accounts | Nominal GDP quarterly |
| DE_ECB_RATE | `BBIN1/M.D0.ECB.ECBREF.EUR.ME` | Central Bank Rates | ECB main refinancing rate |
| DE_BUND10 | `BBMF1/M.D0.EUR.GOVT.WYWG.Y10.BID` | Financial Market | 10Y Bund yield |
| DE_EURUSD | `BBEX3/M.USD.EUR.BB.AC.A01` | Exchange Rates | EUR/USD monthly average |
| DE_HOUSE_IDX | `BBDY1/M.B30.Y.G800.R0400.A` | Residential Property | House price index |
| DE_MORTGAGE | `BBIB1/M.DE.B.S.DNB.KK.A.17.N3.11A` | Interest Rates | 10Y mortgage rate (suspect — returns 2003 data) |

## Destatis Table Codes

| Metric ID | Table Code | Description |
|-----------|-----------|-------------|
| DE_INDPRO | 42153-0001 | Industrial Production Index |
| DE_ORDERS | 42131-0001 | Factory Orders |
| DE_RETAIL | 45212-0004 | Retail Sales (Umsatz im Einzelhandel, Monate) |
| DE_CPI | 61111-0002 | Consumer Price Index |
| DE_PPI | 61241-0001 | Producer Price Index (failing: needs classifying var PRE001) |
| DE_IMPORT_PRICE | 61411-0002 | Import Price Index (Index der Einfuhrpreise, Monate) |
| DE_LABFORCE | 13211-0001 | Labor Force |
| DE_AVG_WAGE | 62321-0002 | Average Wage (currently failing) |
| DE_COMPLETIONS | 31121-0001 | Housing Completions |
| DE_PERMITS | 31111-0001 | Building Permits |
| DE_POP | 12411-0001 | Total Population |
| DE_GINI | 63081-0003 | GINI Index (currently failing) |
| DE_MEDIAN_INC | 63081-0002 | Median Household Income (currently failing) |
| DE_POVERTY | 63081-0001 | Poverty Rate (currently failing) |
| DE_BIRTH_RATE | 12612-0001 | Birth Rate |
| DE_LIFE_EXP | 12621-0001 | Life Expectancy (currently failing) |
| DE_MORTALITY | 12613-0001 | Mortality Rate |
| DE_SUICIDE | 23211-0004 | Suicide Rate (currently failing) |

## Notes

- **Trade tables (51000-series):** Require classifying variables — cannot be fetched as plain tables. Dropped from config until alternative codes found.
- **Dropped Bundesbank series:** 2Y Bund, Bund spread, DAX, VDAX, EUR exchange index, Core CPI, WTI Oil, capacity utilization, disposable income, savings rate, consumer spending, household debt, government fiscal aggregates — not found in new Bundesbank SDMX API.
- **DE_PPI (61241-xxx):** All tables in this series require classifying variable PRE001. No plain "total" table available. Metric fetches will fail until a workaround is found.
- **DE_AVG_WAGE (62321-0001/0002):** Code 98 (table too large for dialog API). No alternative table identified. Metric deferred.
- **DE_GINI / DE_MEDIAN_INC / DE_POVERTY (63081-xxx):** All return code 104 (no objects found). Tables may have been reorganized or replaced in GENESIS. Metrics deferred.
- **BA fuer Arbeit (infosysbub):** API now returns HTTP 403. Requires OAuth2 client credentials. All 5 employment metrics (DE_UNRATE, DE_EMPRATE, DE_JOLTS, DE_BAX, DE_KURZARBEIT) blocked until credentials obtained.
- **ZEW adapter (as of 2026-04-18):** Migrated from old `sentix.csv` URL (404) to `highchart-data.csv` with row-oriented format. "Survey ID" row = YYYYMM periods; "Werte Graph 1" = ZEW Expectations; "Werte Graph 2" = Current Situation.