Building a Modern LLM from Scratch: RMSNorm, RoPE, GQA and SwiGLUWhat changed inside the transformer since 2017, why every open model made the same four changes, and a 250-line model you can train on Shakespeare tonight.Oct 13, 2026·43 min read