Why Can’t LLMs Count the R’s in “Strawberry”?
Why LLMs famously miscounted the r’s in strawberry, what tokenization hides from the model, and why newer models get it right.
KV Cache Explained: Why LLMs Don’t Re-Read Your Whole Prompt Every Token
What the KV cache is, why it makes LLMs fast, why the first token is slow, and why long contexts cost so much memory.
Temperature, Top-k and Top-p: How LLMs Choose Their Next Word
What temperature, top-k and top-p actually do when an LLM picks its next token, how they differ, and which values to use for your tasks.
Why LLMs Get “Lost in the Middle” (And What To Do About It)
LLMs often perform best when relevant information is at the beginning or end of a long prompt, and struggle when it is buried in the middle. Here is why it happens and how to fix it in your RAG systems.