Books
Facts & News
Courses
Agents
Robotics

About
Contact

Courses › How AI Reads › Part 3

1. Words are not the unit
2. Split a sentence yourself
3. The language tax
4. Check yourself
5. What a context window is
6. Fill the window
7. Long isn't the same as read
8. Check yourself
9. Weights, not a diary
10. Memory features, honestly
11. Final check

🔒

Part 3 opens after the part 2 check

Finish the check at the end of part 2 to open this lesson.

Back to the course

Lesson 11 of 11 · Part 3 · Check yourself · 3 min Draft

Final check

Five questions across the whole course, from the first token to what a model keeps afterwards.

Five questions across the whole course. Look back at any lesson if you get stuck.

Check yourself

Get every answer right to finish the course.

1. What does a model actually read? Numbered pieces of text called tokens Whole words, one at a time Each letter on its own

Text is cut into tokens from a fixed menu, and each becomes an ID number before the model sees it.

2. Why can the same question cost more in Burmese than in English? Burmese answers are always longer Models refuse short Burmese questions Its text breaks into many more tokens

Token menus were built mostly from English text. In a 2023 study, Burmese needed about 12 times as many tokens as English.

3. In a very long input, which fact is most likely to be missed? One in the opening lines One in the middle One in the closing lines

Models use information at the start and end of a long input better than information in the middle.

4. Without a memory feature, what carries over into a new chat? The full previous chat A short summary of the old chat Your name and preferences Nothing from the old chat

A new chat starts with none of the old conversation in its window. Yesterday's chat is gone unless you paste it back in.

5. What changes a model's weights? A very long chat A memory feature Training Pasting in a document

Training is the only thing that sets the weights. Chats, saved notes and pasted text all go into the window instead.

← Memory features, honestly Finish: back to the course →

Books

The Model Race
All books
Errata & corrections

Learn

Facts & News
Courses

Build

Agents
Robotics

Lab

About

Company

Contact