FundamentalsPrompt Caching: Stop Paying for the Same Tokens Twice
Prompt caching reuses the prefill work for a repeated prefix, cutting cost and first-token latency. The whole technique comes down to one ordering rule.
ReadNotes on models, agents, and what inference costs
Practical writing for developers building with large language models — how they work, how to pick one, and how to keep the bill predictable.
Fundamentals
AI AgentsPrompt Engineering for Coding Agents Is Mostly Context Engineering
Clever phrasing barely moves the needle. What actually improves agent output is what you put in front of it, in what order, with what constraints.
Read
AI AgentsPrompt Injection: Why Your Agent Is a Confused Deputy
An agent that reads untrusted content can be instructed by it. The fix is not a better prompt — it is treating the model as an untrusted client.
Read
FundamentalsQuantization Explained: Big Models on Smaller Hardware
Quantization shrinks model weights to fewer bits so they fit in less memory. Here is how the formats differ, what accuracy you lose, and how to pick one without guessing.
Read
ModelsQwen 3.5 for Coding: Picking a Size You Can Actually Run
The Qwen line is the only open family that spans 27B to 397B under Apache 2.0. A guide to choosing a size, reading its benchmarks, and self-hosting economics.
Read
FundamentalsRAG vs Long Context: Retrieval Did Not Become Obsolete
Large context windows were supposed to kill retrieval. They did not. Here is how the two actually compare on cost, accuracy and latency — and when to use each.
Read
GuidesRate Limits and Retries: Backoff That Does Not Make It Worse
Token buckets, jitter, retry budgets and circuit breakers for LLM APIs — how to stay under the limit instead of discovering it, and why naive retries amplify outages.
Read
AI AgentsReAct vs Plan-and-Execute: Two Ways to Run an Agent Loop
ReAct decides one step at a time. Plan-and-execute commits to a plan first. The choice changes cost, latency, recoverability and how failures look.
Read
FundamentalsReasoning Models Explained: When Thinking Longer Helps
Reasoning models spend extra tokens working before they answer. Here is what that changes mechanically, which tasks it helps, and where it is just an expensive delay.
Read
GuidesRoo Code Setup: Custom Endpoint and Profiles Per Mode
Configure Roo Code against an OpenAI-compatible base URL, then use API configuration profiles to give each mode its own model. Includes the tool-calling caveat.
Read
GuidesRunning LLMs in CI Without Flaky, Expensive Builds
Putting a model call inside CI adds a non-deterministic network dependency to your build. How to keep it cheap, secret-safe on forks, and incapable of blocking a merge.
Read
Cost & PricingSelf-Hosting vs a Managed API: Find Your Break-Even
Running open weights on rented GPUs has a break-even volume. Here is how to compute yours from throughput and utilisation, and why most teams never reach it.
ReadShowing 697–708 of 731 articles