acilox / solutions / supply-chain-warehouse
📦 Data Engineering

Supply Chain Warehouse

Kimball-style Snowflake warehouse unifying SAP ERP, Oracle WMS, MQTT truck IoT, and carrier APIs into conformed facts, SCD-2 dimensions, and an OTIF / fill-rate KPI engine.

OTIF, fill, lead, backorder
KPIs computed
1m / 5m / 1h
IoT aggregation windows
2.5σ
Anomaly threshold
2–8 °C
Cold-chain breach band

Why this exists

Supply-chain operations need a single analytical surface across ERP, WMS, in-transit telemetry, and third-party signals to answer questions like on-time-in-full, fill rate and lane-level performance. The Kimball build for that is well-understood but rarely shown end-to-end. This is the full pattern — conformed dimensions across order and shipment grain, SCD-2 history for slow-moving attributes, IoT windowed aggregation, and a KPI engine on top of the marts.

How it's wired together

sources (driven from sources.yaml)

z ≥ 2.5σ /
cold-chain breach

SAP HANA
(ERP orders)

Oracle WMS
(picking)

MQTT
(truck IoT)

weather API

carrier API

Airflow DAG factory
(one YAML → many DAGs)

IoT windowed aggregation
1m · 5m · 1h

alerts

staging
(Snowflake)

SCD-2 MERGE
(supplier · warehouse · product)

Snowflake warehouse
fact_order · fact_shipment · fact_telemetry

KPI engine
OTIF · fill · lead · backorder

BI dashboards

Supply Chain Warehouse architecture overview

How it works, end-to-end

  1. Multi-source ingest, single sources.yaml

    SAP HANA (ERP orders), Oracle WMS (picking events), MQTT truck telemetry, weather and carrier APIs all ingest via a DAG factory that reads one sources.yaml. Adding a source is a config change, not new DAG code.

  2. Conformed dimensions + SCD-2 history

    Supplier, warehouse, and product dimensions are SCD-2 (effective_from / effective_to) via MERGE. Every fact grain (order, shipment, telemetry, weather) shares the same dimension keys.

  3. IoT windowed aggregation

    Truck telemetry rolls up at 1-minute, 5-minute and 1-hour grain. Z-score anomaly detection at 2.5σ flags suspicious sensor drift; cold-chain breach detection alerts on pharma shipments outside 2–8°C.

  4. KPI engine on top of the marts

    Four KPIs computed declaratively from the fact tables — OTIF, fill rate, average lead time, backorder rate — with lane / carrier / SKU drilldowns. Output is consumed by BI without recomputation.

The choices that matter

Decision

DAG-from-config factory

New source = one YAML entry + scheduler restart. No new DAG code. Easier to onboard non-engineers; easier to audit the source surface.

One config bug crashes all DAGs at parse time. Mitigated with a CI lint that loads the YAML through the factory before merge.

Decision

SCD-2 on slow-moving dimensions only

Supplier change-of-address, warehouse capacity revisions, product reclassification — all SCD-2. High-volume fact rows are immutable inserts. Right balance of history vs. write cost.

Decision

KPI engine reads from marts, not raw facts

KPI definitions are version-controlled and operate against pre-aggregated marts. Recompute is cheap; reproducing a historical KPI is one query, not a re-run.

Decision

Oracle WMS as an interface stub

Real WMS deployments hit hand-written SQL views maintained by the operations team. The interface in this repo documents the contract; production swaps the implementation behind it.

Built on

SnowflakeSAP HANAOracleAirflow 2.xMQTTPython 3.11KimballGreat Expectations