How Large Language Models Work
Open the black box of large language models: how they predict text, why they sometimes make things up, and the handful of settings that change how they answer.
Temperature and sampling
The dial between focused and creative When a model predicts the next token, it does not settle on a single option — it produces a whole ranked list of candidates with probabilities. “The sky is” might be 80% “blue”, 5% “grey”, 2% “falling”, and so on down the list. Temperature is the setting that decides […]
Cost, latency, and picking a model
Every provider offers a range, and the biggest is rarely the right default Models come in tiers — small and fast, mid-range, and large and capable — and the price between the ends of that range is often an order of magnitude. Reaching for the largest by reflex is the most common way to spend […]
Reading a model announcement without being sold to
New models ship constantly. Most announcements will not change your work Keeping up is exhausting and mostly unnecessary. What you need is a filter that turns a launch post into a decision in five minutes. The four things that actually matter to you Price per token, input and output. This is the number most likely […]