FundamentalsMulti-Head Latent Attention Explained
MLA compresses the KV cache into a small latent vector instead of sharing heads. Why it trades compute for memory, and which models bet on it.
ReadNotes on models, agents, and what inference costs
Practical writing for developers building with large language models — how they work, how to pick one, and how to keep the bill predictable.
How large language models actually work, in plain terms.
Fundamentals
FundamentalsNeedle-in-a-Haystack Tests: What They Prove and What They Miss
A perfect needle-in-a-haystack chart says a model can find one planted sentence. It says very little about whether it can reason over your documents.
Read
FundamentalsPaged Attention: Virtual Memory for the KV Cache
Paged attention stores the KV cache in fixed-size blocks instead of one contiguous slab, which is what lets a server hold far more concurrent sessions.
Read
FundamentalsPlanning Storage for Self-Hosted Model Weights
How to size disk for open-weight models: what drives checkpoint size, why quantisations and versions multiply it, and why storage is rarely the real constraint.
ReadFundamentalsPositional Encoding: How a Transformer Knows Token Order
Attention is order-blind by default. How position gets injected, why the method decides how far context can stretch, and what you see when it fails.
Read
FundamentalsPretraining vs Post-Training: Where a Model Gets Its Behaviour
Pretraining decides what a model knows. Post-training decides how it behaves. Knowing which stage owns a problem tells you whether prompting can fix it.
ReadFundamentalsRepetition Penalties: Why They Break Code Generation
Frequency and presence penalties suppress loops by punishing tokens the model already used. On code that punishes syntax, and the loop was rarely the real bug.
Read
FundamentalsResidual Connections: Why Deep Models Train at All
The one-line change that made hundred-layer networks possible, why the residual stream behaves like a shared bus, and what it explains about pruning.
ReadFundamentalsRLHF Explained: Training on Comparisons Instead of Answers
Reinforcement learning from human feedback shapes the qualities nobody can write down. The three-stage pipeline, and the failure modes you see as a user.
Read
FundamentalsRotary Embeddings (RoPE) Explained With Clock Hands
RoPE encodes position by rotating vectors rather than adding to them. Why that gives relative distance for free, and how it made 1M context possible.
ReadFundamentalsScaling Laws Explained: What Predicts a Model Getting Better
Scaling laws describe how loss falls as compute, data and parameters grow. What they actually claim, where they stopped applying, and why it matters to you.
Read
FundamentalsSelf-Consistency Decoding: Ask Five Times, Take the Majority
Sampling several answers and voting beats a single answer on some tasks and is pure waste on others. The mechanism, the cost, and when to reach for it.
ReadShowing 25–36 of 72 articles