LLM Field Notes

A simple objective demands deep ability

“Try seriously to predict the next word and grammar stops being enough.” That is the heart of it.

• Predicting “1234 + 5678 = ___” requires arithmetic
• Continuing “The butler did it, because ___” requires reasoning
• Answering “this code fails because ___” requires understanding programs

So an objective as crude as “lower the prediction error” ends up demanding world knowledge and reasoning as a by-product. That is what the intelligence really is.

Four stages of processing

Why they suddenly got smart

Scaling laws. Increase data, parameters and compute and performance improves in a remarkably predictable way. Past a certain threshold, abilities nobody trained for — multi-step reasoning, for instance — appear all at once (emergence).

A word’s coordinates are fixed, like a dictionary entry. But the bank of a river and the bank that holds your money are the same string. Fixed coordinates cannot tell them apart. Something has to let a word rebuild its own meaning from its surroundings.

The three roles: Q, K, V

Three steps of arithmetic

1. Take the dot product of one word’s Q with every word’s K (affinity). 2. Run softmax so the weights sum to one. 3. Take the weighted average of every word’s V and make that your new vector.

“Context determines meaning” is, concretely, this weighted average. Surprisingly plain.

Why this design won

• Distance does not matter — a word 100 words back is one dot product away. The older RNNs played telephone and forgot distant words
• It is all matrix multiplication, so it parallelises — historically this was the decisive part. Because GPUs could chew through it, models could grow, and growing is what unlocked the scaling laws

Multi-head, and depth

One average captures one kind of relationship, so dozens of the same operation run in parallel. Probe them and you find heads that track syntactic dependencies, heads that resolve pronouns, heads that hold the topic — a division of labour that emerged on its own. Nobody designed it.

Two footnotes

• Causal mask — for a generative model, seeing future words would be cheating, so they are blocked. An LLM only ever looks left to right
• Quadratic cost — every pair of words is scored, so long contexts get expensive and slow fast. This is where much of today’s research is fought out

Pre-training — the answer is inside the data

Raise the probability of the token that actually came next. That is the whole objective. Because the label is inside the text itself, data can be scaled without human annotation. That was the decisive property.

At this point, though, the model is not usable.

It has the knowledge but does not know that questions are meant to be answered. It is only writing a plausible continuation.

SFT — teaching the form

Train on high-quality human-written question and answer pairs, applying the loss only to the answer. Almost no new knowledge is added; existing ability is simply made retrievable.

RLHF — polishing on preference

Writing model answers is expensive and inconsistent. But choosing whether A or B is better is fast and stable. RLHF exploits that asymmetry.

Step 4 matters: with step 3 alone the model will inevitably find the holes in the reward model (reward hacking). Write at excessive length, add bullet points, flatter the reader — it discovers the tricks that raise the score without adding substance.

RLVR — what a reasoning model really is

Maths and code have answers a machine can check. No reward model is needed; correctness itself is the reward. And since thinking longer raises accuracy, the model spontaneously acquires the habits of writing out long intermediate steps, correcting itself mid-stream, and cross-checking alternative solutions. Nobody taught it those. They fall out of trying to be right more often.

Structural side effects

• Sycophancy — people rate agreeable answers highly, the reward model learns it, and the model becomes ingratiating. It follows from the mechanism
• Alignment tax — pushing safety and politeness harder can cost output diversity and some capability

Cause 1: there is structurally no “I don’t know”

The output is a probability distribution over the whole vocabulary, and low probability is never zero probability. Ask about a paper that does not exist and plausible words still line up with respectable weight. Something has to be emitted, so out comes a sentence that is formally perfect and empty inside.

Cause 2: the knowledge is lossily compressed

Trillions of tokens are squeezed into hundreds of billions of numbers. Compress a JPEG far enough and the outlines survive while the detail collapses.

• Facts that appear often → reinforced repeatedly, robust
• Facts that appear once → buried under surrounding patterns, degraded

The result is an unavoidable bias: strong on famous things, weak on niche proper nouns, figures, dates and citations.

Cause 3: post-training makes it worse

Almost every SFT example is a passage that answers confidently. But the person who wrote it was writing about what they knew. What transfers to the model is only the form of answering.

RLHF compounds this. Raters tend to prefer “Founded in 1897, its first chairman was...” over “I don’t have reliable information” — without checking which is true. Answering confidently is what gets rewarded.

Where it is most likely

What to do

• Supply the material instead of relying on its memory (frame the task as summarise or analyse)
• Receive results in verifiable units (run the code, open the citation yourself)
• Say explicitly “if you don’t know, say so” — this counteracts the pressure from post-training

1. Be specific — the largest single effect

The difference is role, purpose and output format. Note that “you are an excellent X” is routinely overrated; a concrete description of the situation (the reader is a department manager, technical jargon will not land) works far better.

2. Show one example

Tone and format are conveyed far more accurately by an example than by description. Pasting in one good past deliverable lifts the result sharply.

3. Hand over the material

Not “tell me about X” but “read this document and summarise it from the angle of X”. This is the primary defence against hallucination.

4. Order — long material first, instruction last

For separators, ---, headings, or XML tags (<material>...</material>) all work well.

5. Specify the order of thinking

“First list the assumptions and constraints, then propose options.” Ask for the conclusion first and it will be dragged along by whichever direction surfaced first. Exactly like a person.

6. Iterate rather than aiming for one perfect shot

Treat the first output as a draft. Three options, pick one, refine in parts — that lands faster than writing one enormous flawless prompt.

7. Countering sycophancy

• “I am not looking for agreement. Point out the weaknesses.”
• “If this plan fails, what will have caused it?”
• “If my premise is wrong, say that first.”

The third is the best value, because it also heads off hallucination caused by a false premise.

8. Split the conversation

Long threads accumulate a dirty context and keep dragging along a failed direction. When the approach changes, start a new conversation. Given the quadratic cost, needlessly long threads are also slower.

A reusable skeleton

An LLM is fundamentally a box that takes text and returns text; it has no way to touch the world. Three layers sit on top of it. They do not compete — they occupy different layers.

MCP — a common standard for connection

Connections that used to be implemented separately for every AI service, standardised the way USB-C standardised cables. Build one MCP server for GitLab and any AI that speaks the protocol can use it. The server advertises a list of available tools with descriptions; the LLM reads it and decides which to call.

Skill — a procedure read only when needed

In substance it is a Markdown document. The clever part is the design: only the description is always visible, and the body is read the first time a matching request arrives. Loading everything at all times would crowd the context — the same idea as a library index.

Agent — deciding the next step from the result

The decisive difference is that it handles the unexpected on its own. A test failed, the situation is not what was assumed — whether it can change course there is what separates it from an automation script.

The three working together

MCP is the hands, the skill is the knowledge, the agent is the will.

Things you will certainly hit in practice

• Permissions — it can really delete files and really send messages. Confirmation steps have to be designed in
• Prompt injection — external data may contain “ignore all previous instructions”. Treat anything from outside as data, never as instructions. Because attention refers to strings in the context without distinguishing their origin, this remains hard to defend completely
• Reliability compounds — 95% per step becomes 36% over twenty steps. The longer the loop, the more verification matters

The specifications published with a model fall into six families: size, capacity, precision, variant, scores and price.

Size — B

B = billion, the parameter count. A 7B model holds seven billion numbers. This is precisely the capacity of the lossily compressed knowledge from chapter IV.

• Notation like 671B-A37B — 671B total parameters, of which 37B are actually active per token (A = active). Capability tracks the total, speed tracks the active count. 8x7B is the same family
• T (trillion) — training tokens. At equal B, more training wins

Context length — K / M

128K means 128,000 tokens. This is where chapter II’s quadratic cost bites. Two caveats: input and output are usually counted separately, and the advertised figure is not the usable figure — models are known to lose material in the middle of a long context. It fits; that does not mean it is used accurately.

Precision and quantisation — the crux of running locally

Q4_K_M is a GGUF quantisation scheme; the K family varies precision block by block, and the trailing S/M/L marks the trade-off between size and quality. When in doubt, Q4_K_M or Q5_K_M. Below 4-bit the degradation becomes abruptly visible.

Suffixes that mark the variant

Benchmarks

Contamination — benchmark items leaking into training data — is always suspected, and the numbers are self-reported. Different measurement conditions make comparison meaningless anyway. For practical work look at SWE-bench; for how a model feels, look at Elo.

Licence and price

Open weights is not open source. The weights are published while the training data and code are not, and commercial use often carries conditions. Apache 2.0 or MIT means effectively no restrictions.

• $ per 1M tokens — input and output are priced separately. Output is typically three to five times input
• Cache pricing — where the same prefix repeats, this dominates the real bill
• TPS / TTFT — perceived speed depends more on time to first token

Parameters you set at run time

These “parameters” are all settings for how to pick from the probability distribution. Think back to the diagram on the cover.

In practice touching temperature alone is enough. Move it together with top_p and the behaviour becomes unreadable.

So what do you actually look at