FundamentalsAttention Mechanisms Explained Without the Linear Algebra
What attention actually computes, why it made transformers work, and why its cost scaling explains almost every practical limit you hit with long context.
ReadNotes on models, agents, and what inference costs
Practical writing for developers building with large language models — how they work, how to pick one, and how to keep the bill predictable.
How large language models actually work, in plain terms.
Fundamentals
FundamentalsBatch Size and Throughput: The Trade Behind Every Token Price
Batching is why per-token prices are low and why latency varies under load. How it works, where it stops helping, and what it means for your requests.
Read
FundamentalsBeam Search vs Sampling: Why Chat Models Do Not Search
Beam search finds higher-probability text and worse text. Why sampling won for open-ended generation, and where search-like decoding still earns its place.
Read
FundamentalsByte-Pair Encoding Explained: How Tokenizers Are Built
BPE is a compression algorithm that became the standard way to split text for language models. How it is trained, what it produces, and why it behaves oddly.
ReadFundamentalsChain of Thought: Why Thinking Out Loud Actually Helps
Asking a model to reason step by step measurably improves accuracy on some tasks and wastes tokens on others. The mechanism, and when it is worth the cost.
ReadFundamentalsConstitutional AI: Alignment From Written Rules, Not Labels
Instead of paying humans to rank thousands of responses, write the principles down and have the model apply them. How the method works and where it strains.
ReadFundamentalsContext Length vs Effective Context: The Number You Can Use
A model advertising a one-million-token window does not reliably use one million tokens. The gap between the spec sheet and what actually works.
Read
FundamentalsContinuous Batching: How Servers Keep GPUs Busy
Continuous batching lets finished requests leave a batch and new ones join mid-flight. It is why modern inference servers hold high load without stalling.
ReadFundamentalsData Contamination in Training: Why Scores Overstate Skill
How benchmark data leaks into training sets, why it is hard to prove, and what a contaminated score actually costs you when you pick a model for real work.
ReadFundamentalsDPO vs RLHF: Two Routes From Preferences to Model Behaviour
Direct preference optimisation drops the reward model and the RL loop. What that simplification buys, what it costs, and why open-weight work leans on it.
Read
FundamentalsExpert Routing in MoE: How a Token Picks Its Path
The router decides which experts see each token, and that one small network shapes quality, throughput and why identical prompts can behave differently.
Read
FundamentalsFlash Attention Explained: Same Maths, Far Less Memory
Flash attention makes long context practical by never writing the score matrix to memory. What it changes, what it does not, and where you feel the difference.
ReadShowing 1–12 of 72 articles