Clustering

Unsupervised grouping. The hard part isn’t the algorithm — it’s that there’s no ground truth, so “is this good?” has no clean answer. Interviewers probe exactly that.

k-means

Pick k centroids, assign each point to the nearest, recompute centroids as the mean of their members, repeat until stable (Lloyd’s algorithm).

from sklearn.cluster import KMeans

km = KMeans(n_clusters=5, n_init=10, random_state=42).fit(X_scaled)
labels, centers = km.labels_, km.cluster_centers_

Assumptions baked in, and each is a way it fails:

That last one matters: k-means has no concept of noise. Outliers get assigned to a cluster and drag its centroid.

Scaling is mandatory — it minimises Euclidean distance.

n_init matters. Lloyd’s algorithm converges to a local optimum that depends on initialisation. Running it multiple times and keeping the best inertia is why n_init=10 (or "auto") is the default. k-means++ initialisation, also default, spreads initial centroids to reduce bad starts.

Choosing k

inertias = [KMeans(n_clusters=k, n_init=10).fit(X).inertia_ for k in range(2, 15)]

• Elbow method — plot inertia against k and look for the bend. Often ambiguous; be honest about that.
• Silhouette score — how close each point is to its own cluster versus the nearest other one, in [-1, 1]. More principled than the elbow.
• Domain constraints — “marketing can run five campaigns” is frequently the real answer, and saying so is a strength, not a cop-out.

from sklearn.metrics import silhouette_score
silhouette_score(X, labels)     # higher is better; ~0 means overlapping clusters

MiniBatchKMeans

For large data, MiniBatchKMeans uses random batches per iteration. Much faster, slightly worse. Standard for millions of rows.

DBSCAN

Density-based: cluster points that have at least min_samples neighbours within eps, expand transitively, label the rest as noise.

from sklearn.cluster import DBSCAN

db = DBSCAN(eps=0.5, min_samples=5).fit(X_scaled)
# labels_ == -1 means noise

What it buys you over k-means:

• No need to specify the number of clusters.
• Arbitrary shapes — it will find two interleaved crescents that k-means cannot.
• Explicit noise label, which makes it usable for outlier detection.

The cost: eps is hard to choose and the result is very sensitive to it. The usual heuristic is a k-distance plot — sort each point’s distance to its min_samples-th neighbour and look for the knee.

DBSCAN also struggles when clusters have different densities, since one global eps can’t fit both. HDBSCAN fixes that by varying density thresholds, and it’s the better default in 2026:

from sklearn.cluster import HDBSCAN      # sklearn >= 1.3
h = HDBSCAN(min_cluster_size=15).fit(X_scaled)

min_cluster_size is far more intuitive to set than eps, which alone makes HDBSCAN the more practical choice.

Hierarchical clustering

Build a tree by repeatedly merging the closest clusters (agglomerative). Cut the dendrogram at whatever height gives the granularity you want.

from sklearn.cluster import AgglomerativeClustering

agg = AgglomerativeClustering(n_clusters=None, distance_threshold=1.5,
                              linkage="ward").fit(X_scaled)

Strength: the dendrogram shows structure at every granularity, which is genuinely informative. Weakness: O(n^2) memory and O(n^3) time in the naive form, so it’s limited to modest datasets.

Gaussian mixture models

Model the data as a mixture of Gaussians, fit by EM. Unlike k-means it gives soft assignments — a probability of belonging to each cluster — and it handles elliptical clusters because each component has its own covariance.

from sklearn.mixture import GaussianMixture

gm = GaussianMixture(n_components=5, covariance_type="full").fit(X)
probs = gm.predict_proba(X)
bic = gm.bic(X)     # use BIC/AIC to choose n_components - a real model-selection criterion

k-means is effectively a GMM with spherical, equal-variance components and hard assignment. Being able to say that connects the two cleanly.

Evaluating without labels

Internal metrics reward the geometry the metric happens to prefer — silhouette favours spherical clusters, so it will rate k-means above DBSCAN on crescent-shaped data even when DBSCAN is obviously right. Always look at the clusters, projected to 2D with PCA or UMAP, before trusting a number.

The honest senior answer to “how do you know the clustering is good”: you check whether the clusters are useful and interpretable for the decision they feed. Profile each cluster’s feature distributions and see whether they tell a coherent story.

Practical notes

• Clustering high-dimensional data directly rarely works — distance concentration again. Reduce first (PCA to ~50 dims, or UMAP to 2-10), then cluster. Standard pipeline for embeddings.
• Categorical features don’t work with Euclidean distance. Use k-modes, Gower distance, or embed them.
• Clustering is not stable. Re-run on new data and cluster identities shift. If downstream systems depend on cluster IDs, you need an explicit assignment rule for new points and a versioning strategy.

Interview angle

• “How does k-means work and what does it assume?” — alternate assignment and centroid update until convergence. Assumes spherical, similarly sized, similarly dense clusters, meaningful Euclidean distance, and no outliers. Each assumption is a failure mode.
• “How do you choose k?” — elbow on inertia, silhouette score, BIC if using a GMM, or a domain constraint. Say that the elbow is often ambiguous and that domain constraints frequently decide it in practice.
• “k-means vs DBSCAN?” — k-means needs k, finds spherical clusters, assigns every point. DBSCAN infers cluster count, finds arbitrary shapes, and labels noise explicitly — but is sensitive to eps and struggles with varying densities. HDBSCAN handles the density variation and takes a more intuitive parameter.
• “How do you evaluate clustering without labels?” — internal metrics (silhouette, Davies-Bouldin) with the caveat that they encode geometric preferences; visual inspection after dimensionality reduction; and, most importantly, whether the clusters are interpretable and useful for the downstream decision.
• “Why does k-means give different results across runs?” — Lloyd’s algorithm converges to a local optimum dependent on initialisation. Mitigate with k-means++ and multiple restarts (n_init).
• “You want to cluster 1M text embeddings at 1536 dimensions. Approach?” — reduce first (PCA or UMAP), then MiniBatchKMeans for speed or HDBSCAN if you want noise handling and no fixed k. Clustering directly in 1536-D suffers from distance concentration.