My bachelor’s thesis “Intelligent Query Routing for Polyglot Persistence” investigates whether machine learning can predict, for each individual database query, which of two databases will answer it faster. The realistic application domain is an e-commerce platform with around 1.14 million records.
Polyglot persistence as the foundation
The same dataset is modelled in two worlds: relational in PostgreSQL (3NF with an EAV part) and document-oriented in MongoDB (embedded documents). Both run under identical, fair conditions (the same CPU and RAM limits in containers) so the comparison holds up.
Approach
- Benchmark: 18 query templates produce 2,000 equivalent SQL/MongoDB pairs. Before every measurement, an automatic equivalence check confirms both sides return exactly the same result (2,000/2,000).
- Features: 8 features per query, derived from the SQL AST (sqlglot) and the
EXPLAINplan (estimated rows, joins, tables and more). - Models: Random Forest, XGBoost and MLP, trained with a stratified 80/20 split and 10-fold cross-validation, backed by holdout, Wilson confidence intervals and bootstrap.
- Evaluation: routing strategies compared, t-test and Wilcoxon, effect sizes, regret, and feature importance via Gini, permutation and SHAP.
Results
- Intelligent routing is measurably faster than any single fixed database (+6.64%, regret just 0.12%).
- The Random Forest predicts the faster database with 96.5% accuracy.
- Two of the four hypotheses were falsified (estimated row count dominates over join and table count; Random Forest and XGBoost are equivalent). A refuted hypothesis is a valid result here: what counts is the methodology, not the number of confirmed assumptions.
- A CLI prototype runs the full chain live (feature extraction → model → target database) at around 3.2 ms overhead.
The whole setup is built to be reproducible: fixed seeds, versioned configuration, one commit per measurement step and figures generated only by script.
Technically a Python evaluation pipeline (pandas, scikit-learn, XGBoost, sqlglot) against PostgreSQL and MongoDB in Docker, documented throughout in a lab journal.