Classical ML

Still what wins on tabular data in 2026. Gradient boosting beats deep learning on most structured problems, and interviews for AI-facing backend roles routinely check that you know the classics rather than only the LLM layer.

Linear models

#	File	Why it’s asked
01	01_linear_regression.md	assumptions, and which ones matter for prediction vs inference
02	02_logistic_regression.md	the default baseline; why cross-entropy not MSE; threshold != model

Trees and ensembles

#	File	Why it’s asked
03	03_decision_trees.md	the base learner everything else is built from
04	04_random_forest_bagging.md	bagging attacks variance — why the trees are deep
05	05_gradient_boosting.md	the one that wins on tabular data; XGBoost vs LightGBM vs CatBoost

Distance and probability based

#	File	Why it’s asked
06	06_svm.md	the kernel trick
07	07_knn_naive_bayes.md	kNN is vector search; Naive Bayes as the fast text baseline

Unsupervised

#	File	Why it’s asked
08	08_clustering.md	k-means assumptions, DBSCAN/HDBSCAN, evaluating without labels
09	09_dimensionality_reduction.md	PCA vs UMAP, and compressing embeddings to cut RAG cost

The two comparisons worth memorising

Bagging vs boosting. Bagging trains independent deep trees in parallel and averages them to cut variance; more trees never overfit. Boosting trains shallow trees sequentially, each fitting the last one’s errors, to cut bias; more trees eventually do overfit, so early stopping is mandatory. Same ensemble family, opposite mechanism, opposite base learner.

Why trees still beat neural networks on tabular data. Heterogeneous feature types, piecewise-constant relationships, robustness to uninformative features, and far lower data and tuning cost. Deep learning wins when you need to embed high-cardinality relations or fuse tabular data with text or images.