Training and fine-tuning
From pretraining through post-training to the adaptation you’d actually do.
| # | File | Covers |
|---|---|---|
| 01 | 01_pretraining.md | next-token objective, data pipeline, scaling laws, base vs instruct |
| 02 | 02_sft_instruction_tuning.md | SFT, data quality over quantity, LoRA defaults, synthetic data |
| 03 | 03_rlhf_and_preference_optimization.md | PPO → DPO → GRPO → RLVR, and why the field moved |
| 04 | 04_quantization.md | INT4/FP8, GPTQ/AWQ, QLoRA, quality trade-offs |
| 07 | 07_fine_tuning_vs_rag.md | the decision most teams get wrong |
The three that get asked
Compute-optimal is not inference-optimal. Chinchilla optimises training cost. If you serve a model heavily, over-train a smaller one — you buy cheaper inference forever.
GRPO dropped the critic. PPO needs a value network to compute advantages; GRPO samples a group of responses to the same prompt and uses the group mean as the baseline. Half the memory, still on-policy. That’s why reasoning training converged on it.
Fine-tuning teaches form, not facts. Knowledge in weights can’t be updated or cited. If the ask is “answer from our documents”, it’s retrieval.