Choosing an ML framework

The 2026 landscape, and what you’d actually reach for. Verified 2026-08.

The map

Two things this table encodes that are worth saying out loud: scikit-learn is still the default toolset for classical modelling and clean pipelines, and gradient boosting still dominates structured data. Reaching for a neural network on tabular data is usually the wrong instinct — see ../02_classical_ml/05_gradient_boosting.md.

scikit-learn

The reason it survives isn’t the algorithms — it’s the API contract. fit/predict/transform is consistent across everything, which makes composition trivial:

from sklearn.pipeline import Pipeline
from sklearn.compose import ColumnTransformer
from sklearn.model_selection import GridSearchCV

pipe = Pipeline([
    ("pre", ColumnTransformer([...])),
    ("model", HistGradientBoostingClassifier()),
])

search = GridSearchCV(pipe, param_grid, cv=5, scoring="average_precision")

The Pipeline isn’t a convenience — it’s what makes cross-validation honest, because every transform refits inside each fold. That’s the strongest single argument for using it, and it’s a correctness argument rather than a style one. See ../03_feature_engineering/04_data_leakage.md.

HistGradientBoostingClassifier is sklearn’s own LightGBM-style implementation — native categorical support, handles NaN, and fast. Often good enough without adding a dependency.

PyTorch vs TensorFlow vs JAX

PyTorch is the default answer — it’s what nearly all published research and every LLM library targets. TensorFlow retains an edge in mature mobile and edge deployment. JAX is for people who want vmap/grad/jit composition and are comfortable with functional purity.

The pragmatic framing: you rarely choose. You use what your pretrained model was published in, and that’s overwhelmingly PyTorch.

Hyperparameter tuning

import optuna

def objective(trial):
    params = {
        "max_depth": trial.suggest_int("max_depth", 3, 10),
        "learning_rate": trial.suggest_float("learning_rate", 1e-3, 0.3, log=True),
        "subsample": trial.suggest_float("subsample", 0.5, 1.0),
    }
    return cross_val_score(LGBMClassifier(**params), X, y,
                           cv=5, scoring="average_precision").mean()

study = optuna.create_study(direction="maximize")
study.optimize(objective, n_trials=100)

Optuna’s advantage over grid search is pruning — it abandons unpromising trials early — and Bayesian search over the space rather than exhaustive enumeration. Note log=True for learning rate: search rates on a log scale, always.

Grid search is defensible only for two or three parameters with few values. Beyond that it wastes most of its budget.

The honest framework advice

• Don’t add a framework to avoid writing thirty lines. An agent loop, a training loop, or a retry wrapper are all things you can write and debug.
• Do use a framework for the things that are genuinely hard: distributed training, durable execution, serving throughput.
• Consistency beats optimality. A team fluent in one stack ships faster than one that picked the theoretically best tool for each task.
• Check what the pretrained weights are in. That decides more than any feature comparison.

Interview angle

• “Which framework for a tabular classification problem?” — scikit-learn for the pipeline and evaluation machinery, with gradient boosting as the model. Neural networks lose on structured data, and sklearn’s Pipeline is what makes your cross-validation honest.
• “PyTorch or TensorFlow in 2026?” — PyTorch by default: it’s what research and the entire LLM ecosystem target. TensorFlow retains an advantage in mature mobile/edge deployment. In practice you use whatever your pretrained checkpoint was published in.
• “What is JAX for?” — composable function transforms (grad, jit, vmap, pmap) with functional purity, TPU-first. Excellent for research and scientific computing; less mature deployment tooling.
• “Why is Pipeline more than a convenience?” — it makes preprocessing refit inside every cross-validation fold, which is the difference between an honest estimate and one contaminated by test statistics.
• “Grid search or Optuna?” — Optuna past a handful of parameters: Bayesian search plus early pruning of unpromising trials uses the budget far better. Search learning rates on a log scale either way.
• “When would you avoid adding a framework?” — when it replaces code you could write and debug in thirty lines. Frameworks earn their cost on distributed training, durable execution and serving throughput, not on loops.