Architecture · Diagram Design
Open data lake · End-to-end stack
SOURCES
INGEST
DATA LAKE
QUERY
CONSUME
SQL
WRITE
EXT
App servers
Events · logs
EXT
Databases
CDC · exports
FLOW
Apache NiFi
Route · transform
ORCH
Airflow
DAG scheduling
STORE
MinIO
Object store · S3-API
QUERY
Trino
SQL · any format
OLAP
StarRocks
MPP · hot layer
BI
Superset
Dashboards
NB
JupyterLab
Exploration
PROC
Python
Batch · ML
LEGEND
MinIO data lake (focal)
Ingest · query tools
BI · notebooks · pipelines
Write-back path (batch)
Primary query path