token
The prompt, as the model sees it (22 tokens):
What it writes, one token at a time (colour = how sure it was; click any word):
Through the 24 layers: what it would say if it stopped here
lin linear-attention layer · full full-attention layer · green = probability of the final choice
Why 24? Each layer is one more pass of "read the text so far, refine the understanding". The number is a design choice for this model's size: Meta's Llama 3 uses 32 layers at 8B, 80 at 70B and 126 at 405B.
After layer 24, every one of its tokens gets a score in a single multiplication. These are the top 8:
The last prompt token's row in the embedding table (first 12 of numbers):