Services Overview
On-premise · Ubuntu 24.04
Services running
5
All containers up
Storage buckets
3
Bronze · Silver · Gold
Disk used
18 GB
of 581 GB total (3%)
RAM used
~4 GB
of 32 GB total (13%)
Running services
● Live
MinIO Object Store
The file storage system. Holds all raw, cleaned and summarised research data organised in three buckets — Bronze, Silver and Gold. Works like an S3-compatible hard drive inside the server.
● Live
Metabase BI Dashboard
The reporting interface. Institutions, researchers, donors and administrators view KPI dashboards, satisfaction scores, geographic coverage and data quality reports here.
● Live
MinIO Ingest Service
The ingestion bridge. When a researcher submits data on Dacoris, this service polls every 5 minutes, downloads the file, converts it to Parquet and stores it in the Bronze bucket.
● Running
dbt + DuckDB
The transformation engine. Reads Bronze data, builds 12 Silver and Gold KPI models, and writes to DuckDB. Must always be running — Airflow executes dbt every 30 minutes via this container.
● Live
Apache Airflow
The pipeline scheduler. Automatically runs dbt transformations every 30 minutes — rebuilds all 12 Gold models and copies to the read-only Gold database for Metabase. PostgreSQL backend with LocalExecutor.
Metabase dashboards
● Live
Institution Research Dashboard
Survey findings, geographic coverage, gender distribution, satisfaction scores and data quality for all surveys submitted by institution researchers.
● Live
Researcher Dashboard
My submissions, data quality scores, respondent distribution by county and benchmarking against national averages.
● Live
Donor Dashboard
Total beneficiaries reached, counties covered, research areas funded, geographic reach and data submission accountability.
● Live
Admin Dashboard
Pipeline health, ingestion success rates, bronze file registry, researcher activity and full audit trail of all ingested data.
Medallion data layers
🥉 Bronze — raw data
sheet_import.parquet
health_import.parquet
economics_import.parquet
kenya_tertiary_edu.parquet
kenya_economic.parquet
🥈 Silver — cleaned
stg_bronze_files
stg_survey_responses
survey_inventory
🥇 Gold — KPIs
kpi_satisfaction_scores
kpi_geographic_distribution
kpi_demographic_breakdown
kpi_survey_volume
kpi_data_quality
kpi_institution_ranking
kpi_pipeline_health
kpi_researcher_activity
bronze_file_registry
Ingest service — live health
status