When not to use ML

A short file with a high return. Interviewers ask this to see whether you reach for ML reflexively. The strongest answer to “how would you build X with ML” is sometimes “I’d check whether it needs ML first.”

The cases where ML is the wrong tool

The rule is knowable. Tax calculation, business logic, validation, entitlements — if the answer is derivable from rules, write the rules. They’re testable, explainable, instant, and they don’t drift. “Determine whether an order qualifies for free shipping” is an if statement, not a classifier.

You have no labels and no way to get them. ML needs a target. If nobody can say what the right answer was, even in hindsight, there’s nothing to learn from. Fix the labelling problem first.

Errors are unacceptable and unrecoverable. Anything where a wrong answer causes irreversible harm needs either a deterministic system or a human in the loop with the model as an advisor. Frame it as “the model ranks, the human decides”.

The data volume is tiny. A few hundred rows with dozens of features will not generalise. A domain expert’s heuristic will beat it and be easier to defend.

Requirements change faster than you can retrain. If the definition of the target shifts monthly, your training data is describing a world that no longer exists.

Explainability is a hard requirement. Credit decisions, hiring, medical, anything under regulatory scrutiny. Either use an inherently interpretable model (logistic regression, a shallow tree, a scorecard) or accept a significant compliance burden. Post-hoc explanations of a black box are often not sufficient for a regulator.

Nobody will act on the output. If there’s no decision downstream, the model is a dashboard nobody reads.

Start with a baseline, always

Even when ML is right, the sequence should be:

1. Heuristic. A hand-written rule. Ship it, measure it.
2. Simple model. Logistic regression or gradient boosting on a handful of obvious features.
3. Complex model. Only if 1 and 2 leave value on the table.

Each step has to beat the previous one by enough to justify its operational cost. Skipping to step 3 means you never learn how much of the problem was trivially solvable.

from sklearn.dummy import DummyClassifier

# The floor. Any real model must clearly beat this.
baseline = DummyClassifier(strategy="most_frequent").fit(X_train, y_train)

A surprising amount of the time, a well-chosen heuristic captures most of the available value. “We shipped a rule, measured it, and the model added four points” is a much stronger interview story than “we trained a model”.

The 2026 version: does this need an LLM?

The same reflex applies, and it’s more expensive to get wrong because LLM calls cost money per request and add hundreds of milliseconds.

Where LLMs genuinely earn their cost: open-ended language, unstructured-to-structured extraction with high variability, tasks with a long tail no ruleset covers, and anything where the alternative is thousands of labelled examples you don’t have.

The economics question to ask out loud: at what request volume does the per-call cost exceed the cost of building the cheaper thing? At a hundred requests a day, use the LLM and move on. At ten million, a distilled small model pays for itself quickly.

Hybrid is usually the right shape

The best production systems rarely choose. They layer:

• Rules handle the clear-cut majority, deterministically and for free.
• A model handles the ambiguous middle.
• Low-confidence cases escalate to a human.
• Human corrections become training data.

This gives you a system that works on day one, degrades gracefully, improves over time, and has an explainable answer for most cases.

def classify(ticket) -> str:
    if match := RULES.match(ticket):          # cheap, certain, auditable
        return match

    pred, confidence = model.predict_with_confidence(ticket)
    if confidence < THRESHOLD:
        return queue_for_human(ticket)        # and capture the label
    return pred

The escalation threshold is a business decision — it trades human cost against error cost — and saying that explicitly is a seniority signal.

Interview angle

• “When would you not use ML?” — when the rule is knowable, labels don’t exist, errors are unrecoverable, data is tiny, requirements shift faster than retraining, explainability is mandatory, or nobody will act on the output. Lead with “I’d check whether a heuristic solves it” and you’ve already answered well.
• “What baseline do you compare against?” — the existing system if there is one, otherwise majority class or the mean, plus a simple model on obvious features. Establish it before building anything, so you can quantify what the complexity actually bought.
• “Product wants an LLM to pull dates out of documents. Thoughts?” — ask about format variability. Fixed formats are a regex. High variability with a long tail is a genuine LLM case. Then ask the volume question, because per-call cost and latency may push toward extracting with an LLM once to build training data, then serving a small model.
• “How do you handle the cases the model gets wrong?” — confidence thresholding with human escalation, and feed the human corrections back as labels. Design the escalation path before deploying, not after the first incident.
• “Your model is 3% better than the rule-based system. Ship it?” — depends on the operational cost. A model needing a feature store, a serving cluster and monitoring has to clear a much higher bar than 3%. Quantify the lift in business terms and compare against the total cost of ownership.