FundamentalsQuantization Explained: Big Models on Smaller Hardware
Quantization shrinks model weights to fewer bits so they fit in less memory. Here is how the formats differ, what accuracy you lose, and how to pick one without guessing.
ReadNotes on models, agents, and what inference costs
Practical writing for developers building with large language models — how they work, how to pick one, and how to keep the bill predictable.
Fundamentals
ModelsQwen 3.5 for Coding: Picking a Size You Can Actually Run
The Qwen line is the only open family that spans 27B to 397B under Apache 2.0. A guide to choosing a size, reading its benchmarks, and self-hosting economics.
Read
FundamentalsRAG vs Long Context: Retrieval Did Not Become Obsolete
Large context windows were supposed to kill retrieval. They did not. Here is how the two actually compare on cost, accuracy and latency — and when to use each.
Read
GuidesRate Limits and Retries: Backoff That Does Not Make It Worse
Token buckets, jitter, retry budgets and circuit breakers for LLM APIs — how to stay under the limit instead of discovering it, and why naive retries amplify outages.
Read
AI AgentsReAct vs Plan-and-Execute: Two Ways to Run an Agent Loop
ReAct decides one step at a time. Plan-and-execute commits to a plan first. The choice changes cost, latency, recoverability and how failures look.
Read
FundamentalsReasoning Models Explained: When Thinking Longer Helps
Reasoning models spend extra tokens working before they answer. Here is what that changes mechanically, which tasks it helps, and where it is just an expensive delay.
Read
GuidesRoo Code Setup: Custom Endpoint and Profiles Per Mode
Configure Roo Code against an OpenAI-compatible base URL, then use API configuration profiles to give each mode its own model. Includes the tool-calling caveat.
Read
GuidesRunning LLMs in CI Without Flaky, Expensive Builds
Putting a model call inside CI adds a non-deterministic network dependency to your build. How to keep it cheap, secret-safe on forks, and incapable of blocking a merge.
Read
Cost & PricingSelf-Hosting vs a Managed API: Find Your Break-Even
Running open weights on rented GPUs has a break-even volume. Here is how to compute yours from throughput and utilisation, and why most teams never reach it.
Read
Cost & PricingSetting an AI Budget for a Small Engineering Team
A bottom-up method for budgeting model spend on a team of five to twenty, including the buffer to hold, the caps to set, and the alerts that matter.
Read
ModelsSmall vs Large Models: Why Parameter Count Stopped Meaning Much
Mixture-of-experts split model size into total and active parameters, and only one of them predicts your bill. How to think about size when picking a model.
Read
FundamentalsSpeculative Decoding: Faster Output, Identical Tokens
Speculative decoding drafts several tokens cheaply and verifies them in one pass, cutting latency without changing the output distribution. How it works.
ReadShowing 373–384 of 404 articles