Stop thinking of temperature as a "creativity slider" for your
Temperature is a scaling factor applied to those logits before the softmax happens. When you crank up the temperature, you're flattening the probability distribution. The high-probability tokens lose some of their lead, and the low-probability tokens get a slight boost. This makes the output more random and unpredictable, but "random" isn't the same as "creative." A model isn't suddenly thinking of a better metaphor; it's just more likely to pick a word that it previously thought was a bad fit.
If you want to actually control the output variety, you need to understand the interplay between temperature, top-k, and top-p.
How sampling parameters actually work
1. Temperature: As mentioned, this scales the logits. A temperature of 1.0 is the default. Below 1.0, the distribution becomes "sharper," making the model more confident and deterministic (good for coding or factual extraction). Above 1.0, it becomes "flatter," increasing diversity but also increasing the risk of hallucinations or gibberish.
2. Top-K Sampling: This is a hard cutoff. If K is set to 50, the model only considers the top 50 most likely tokens and throws everything else away. This prevents the model from picking a completely nonsensical word from the bottom of the list, regardless of what the temperature is.
3. Top-P (Nucleus Sampling): This is a dynamic cutoff. Instead of a fixed number of tokens, it looks at the cumulative probability. If P is 0.9, the model selects the smallest set of tokens whose combined probability reaches 90%. This allows the "pool" of candidate words to expand or shrink based on how confident the model is.
The critical takeaway here is that none of these settings change the actual ranking of the tokens. If "apple" is the most likely token and "banana" is the second, no amount of temperature tweaking will ever make "banana" more likely than "apple." It only changes how much more likely "apple" is.
If you're building an AI workflow and find the output is too boring, don't just slide the temperature to 1.5. Instead, try a deep dive into your prompt engineering. Give the model a specific persona or a constraint that forces it to avoid clichés. That is where actual "creativity" comes from—guiding the model toward a different part of its latent space, rather than just making it roll the dice with less certainty.