Study Notes / 2026 ・ Extra
Vector Field Notes
What it means to turn words into numbers
Published
A phrase used repeatedly in the first two notes
The “coordinates” part was never properly explained
Readers of the first two notes told us that
the vectorisation part did not quite land.
So here is an extra instalment, heavy on diagrams, on that one idea.
Why turn words into numbers
A computer holds nothing but numbers. Characters do have numbers assigned to them, but those numbers carry no meaning. Something else has to supply numbers that do.
A character code is not a meaning
the same animal, and the numbers have nothing to do with each other
“dog”, “hound” and “犬” point at the same animal, yet their codes are unrelated. And two codes that happen to be adjacent say nothing about meaning. A character code is a number for filing, not a scale of meaning.
So we want numbers that represent the meaning itself: numbers that are close when the meanings are close. Preparing such a set of numbers for every word is what vectorisation (embedding) does.
Meaning as a position
The closest analogy is a map. As an experiment, lay out some animals and vehicles along two axes.
dog and cat land close together, the bus far away. That placement is the vector
Size runs left to right, living-ness bottom to top. Dog and cat end up neighbours; the bus sits far off. Those coordinates are the vector. Two dimensions means two numbers, three means three. That is all it is.
What matters is that nobody chose those axes. Read enough text while repeatedly nudging “words appearing in similar contexts belong close together”, and this arrangement falls out on its own. No one taught it an axis called size.
Near “king” you find “queen”, “kingdom”, “coronation”; near “dog” you find “cat”, “walk”, “bark”. How closely words keep company in text becomes how closely they sit in space.
How closeness is measured
On a map you would reach for a ruler. In a space of meaning, what gets measured is usually direction: draw arrows from the origin and look at the angle between them.
the angle between dog and cat is small; the angle to bus is wide.
Measuring closeness by that angle is cosine similarity
Why not length? Longer text produces larger numbers, and a repeated word stretches the arrow. Length moves with quantity; direction is decided by what the text is about. The second is what we want.
Values fall between −1 and 1. Near 1 means almost the same direction, 0 means unrelated, negative means opposed. This is the number behind most search and duplicate-detection features.
Adding and subtracting meaning
Once meaning is a position, something odd becomes possible: meanings can be added and subtracted.
the arrow from man to king and the arrow from woman to queen have almost the same direction and length
The four words form a clean parallelogram: king − man + woman ≈ queen. “Tokyo − Japan + France ≈ Paris” has the same shape.
Again, nobody told the model that king and queen differ only in gender. From how the words are used across enormous amounts of text, the difference in meaning showed up as a direction. No.01 called intelligence a by-product of prediction pushed hard; this parallelogram is a by-product of the same kind.
What one actually looks like
The pictures so far have been two-dimensional. The real thing is a row of several hundred to several thousand numbers.
dog and cat give similar rows; the bus gives a different one (shading is illustrative)
This is where many people get stuck: none of the individual numbers has a name. There is no table saying the third is size and the seventh is living-ness. It is not something a person reads.
That is fine, because what is needed is not reading but measuring. If the angle between two rows can be computed, closeness of meaning is known.
Think of the dimension count (768, 1536 and so on) as the resolution of the map. Two dimensions can only separate size and living-ness; a thousand can hold colour, hardness, period, register, professional field — as many axes as needed.
Moving with context, and uses
The map is not fixed
The map in chapter II is effectively a dictionary. In a real sentence, though, the same word shifts.
the dictionary position is the starting point; the surrounding words rewrite it
That rewriting is attention (No.01, chapter II). The dictionary vector is where a word starts; inside a sentence the coordinates move according to its neighbours. Two stages, one after the other.
What it is good for
The nearest thing to daily work is search: the wording need not match, only the meaning.
Searching for “how do I request annual leave” and getting back “submitting a holiday form” works this way. It is the concrete form of “supply material rather than asking it to recall” from No.01, chapter IV.
- Routing enquiries — pull each towards the nearest category centre
- Duplicate and near-duplicate detection — same content, different wording
- Recommendation — line up what sits nearby
- Images and audio can share the same space (No.02, chapter II)
One caution: a vector cannot be turned back into the original text, but a good deal of the meaning survives. When sending vectors of internal documents to an outside service, treat it as if you were sending the text itself.
§ One page summary
- A character code is only a filing number. Numbers that carry meaning have to be supplied separately.
- Vectorisation places a word on a map of meaning; the coordinates are the vector.
- Nobody picks the axes. They emerge from “words in similar contexts belong close together”, repeated.
- Closeness is measured as direction, not distance — cosine similarity.
- Differences in meaning appear as directions, so arithmetic works.
- The real thing is a row of hundreds of numbers. It need not be readable, only measurable.
- The dictionary position is a starting point; inside a sentence, attention rewrites it.
Vector Field Notes — extra / 2026.09