Lesson 8 of 9 · Part 3 · Play · 2 min Draft
Watch a model type
Watch the same answer arrive at four speeds, from human reading pace to a specialised chip, and see what sets the limit.
The Speed fun-facts page has a live exhibit just below its cards. A short answer streams onto the screen one token at a time, each token a colored chip of text. Below it are four buttons, from human reading pace up to a specialised chip.
Try these three, slowest first:
- Human reading. About 5 tokens a second, roughly how fast you read silently.
- Cloud model. About 80 tokens a second, typical for a large model in the cloud. Notice it is already far ahead of your eyes.
- The fastest button. The answer lands almost at once.
Then read the cards above it. Look for the one that explains the speed limit: each new token means reading all the weights from memory. Next, find the card where 4-bit weights raise that limit four times. Finish with the quiz card about serving many chats at once. It shows how one chip keeps many people’s replies flowing at once.
Quick check
One question to lock it in. It doesn’t count against you.