Why Can’t LLMs Count the R’s in “Strawberry”?
Why LLMs famously miscounted the r’s in strawberry, what tokenization hides from the model, and why newer models get it right.
KV Cache Explained: Why LLMs Don’t Re-Read Your Whole Prompt Every Token
What the KV cache is, why it makes LLMs fast, why the first token is slow, and why long contexts cost so much memory.
Temperature, Top-k and Top-p: How LLMs Choose Their Next Word
What temperature, top-k and top-p actually do when an LLM picks its next token, how they differ, and which values to use for your tasks.