Transfer learning

Training from scratch is now the exception. The default is: take a pretrained model, adapt it. Knowing which adaptation to reach for, and when, is the practical question.

Why it works

Pretraining on a large corpus teaches general structure — edges and textures in vision, syntax and semantics in language. Those representations transfer, so your task only needs to learn the last mile.

The payoff: orders of magnitude less labelled data and compute. A task needing 100,000 labels from scratch may need 500 on top of a good pretrained model.

The spectrum of adaptation

Ordered by cost and by how much you can change:

The rule of thumb: the less data you have and the closer your task is to the pretraining objective, the less you should train. Full fine-tuning on 200 examples overfits catastrophically.

Feature extraction

for p in model.parameters():
    p.requires_grad = False
model.fc = nn.Linear(model.fc.in_features, n_classes)   # only this trains

For text, this is often just: embed everything once, then train a logistic regression.

embeddings = embed(texts)                 # one pass, cached
LogisticRegression().fit(embeddings, y)

Genuinely underrated. It’s fast, needs almost no data, produces a calibrated interpretable classifier, and gives you a strong baseline in an hour. Reach for it before fine-tuning anything.

Fine-tuning, and its two failure modes

Catastrophic forgetting — the model overwrites the general knowledge that made it useful. Mitigations: a low learning rate (1e-5 to 5e-5, roughly 10-100x lower than training from scratch), few epochs (2-4 is typical for text), and freezing early layers.

Discriminative learning rates — lower for early layers, higher for later ones, since early layers hold the most general features:

optimizer = AdamW([
    {"params": model.encoder.parameters(), "lr": 1e-5},
    {"params": model.head.parameters(),    "lr": 1e-3},
])

Overfitting — small datasets plus large models. Early stopping on validation is essential, and evaluate frequently because overfitting can happen within a single epoch.

LoRA and PEFT

The standard way to adapt LLMs. Instead of updating a d×d weight matrix, learn a low-rank update:

W' = W + B @ A        # A is (r, d), B is (d, r), r << d

Only A and B train; W stays frozen. With r=8 on a 4096-dimension matrix, you train 2 * 8 * 4096 ≈ 65k parameters instead of 16.7M — about 0.4%.

from peft import LoraConfig, get_peft_model

config = LoraConfig(
    r=16, lora_alpha=32, lora_dropout=0.05,
    target_modules=["q_proj", "v_proj"],
    task_type="CAUSAL_LM",
)
model = get_peft_model(base_model, config)
model.print_trainable_parameters()          # typically < 1%

Why it matters operationally:

• Memory. Gradients and optimiser state scale with trainable parameters, so the dominant cost disappears. See 03_training_deep_networks.md.
• Multiple adapters, one base. Serve many task-specific adapters against one loaded base model, swapping at request time.
• Mergeable. W + BA can be folded back into W, so inference has zero added latency.

QLoRA goes further: quantise the frozen base to 4-bit and train the adapter in higher precision, which fits large models on a single consumer GPU.

r is the main knob — 8-16 is typical, higher for larger behavioural changes. lora_alpha scales the update; alpha/r is the effective multiplier.

Fine-tune or retrieve?

The most common real decision for LLM work, and often decided wrongly.

Fine-tuning is not how you add knowledge. Facts baked into weights can’t be updated without retraining, can’t be cited, and are easily overwhelmed by the model’s pretraining. If the requirement is “answer questions about our documentation”, that’s retrieval. If it’s “always respond in this JSON structure with this tone”, that’s fine-tuning — or often just a better prompt.

Try in this order: prompt → few-shot → RAG → fine-tune. Each step costs more and moves slower. See ../09_rag_embeddings/ and ../07_training_finetuning/.

Domain gap

Transfer degrades as the target diverges from pretraining. Medical imaging from ImageNet weights, or legal text from general web text, transfers less well.

Continued pretraining bridges it: keep training with the original self-supervised objective on unlabelled domain data before doing supervised fine-tuning. Needs a decent volume of domain text, and it’s the reason domain-specific base models exist.

Interview angle

• “What is transfer learning and why does it work?” — reuse representations learned on a large corpus. Early layers capture general structure that transfers, so your task learns only the last mile, with orders of magnitude less labelled data.
• “You have 500 labelled examples for a text classifier. Approach?” — embed with a pretrained model and fit logistic regression on the frozen embeddings first. It’s fast, hard to overfit, calibrated, and a strong baseline. Full fine-tuning on 500 examples would likely overfit.
• “How does LoRA work?” — freeze the base weights and learn a low-rank update B @ A with rank far below the dimension, so under 1% of parameters train. Gradient and optimiser memory drop proportionally, adapters can be swapped per task, and the update merges back into the weights for zero inference overhead.
• “Fine-tune to add company knowledge to an LLM?” — no. Fine-tuning teaches form, not facts. Knowledge in weights can’t be updated or cited and gets swamped by pretraining. Use retrieval; fine-tune for consistent format, tone or task behaviour.
• “What learning rate for fine-tuning?” — around 10-100x lower than training from scratch, typically 1e-5 to 5e-5 for transformers, for 2-4 epochs. Higher rates cause catastrophic forgetting.
• “Your domain is very different from the pretraining data. Options?” — continued pretraining on unlabelled domain text with the original objective, then supervised fine-tuning. Or find a base model already pretrained in that domain.