Laya Field Notes

As covered in No.03, Jev is an API at $0.042 per million input tokens. Laya has no bill attached, and in exchange you carry the GPU, the operations and the training itself. Same category of model; entirely different thing to take on.

Nandakishor M, CEO of ConvAI Innovations, claims he was working on this a year and a half earlier. Start with the facts.

So both “return a probability rather than prose” and “branch on confidence” were published by the same person more than a year before Jev shipped. The components of what No.03 called the smart if statement were out in the open first.

The System 1 / System 2 split comes from cognitive science and belongs to neither party. Two people arriving at the same idea at the same time is unremarkable in this field. That prior work exists and that copying occurred are different claims, and the second is an accusation rather than an established fact. TypeSafe has not responded publicly.

What is genuinely interesting about the episode: the open side published papers, weights and training data. Because it can be checked, the chapters below can actually check it.

Three things to hold onto: the base, the trick, and the reward.

1. The base — not built from scratch

Laya starts from existing open encoders such as ModernBERT-large. And not the GPT line from No.01 that writes left to right, but the BERT line: a bidirectional encoder that sees both sides at once. If you never have to write, you do not need the machinery for writing.

2. The trick — one hole per option

Lay out the state and the question, then place one [MASK] for each option. A bidirectional encoder gathers, at each hole, how plausible that option is — so one forward pass answers several questions at once. That is where the speed comes from. The word-by-word loop of No.01, chapter I simply does not exist here.

3. The reward — points for honest probabilities

Training uses RLCD, reinforcement learning for calibrated decisions. It scores not only whether the answer was right but whether the probability was honest, using a combination of the log score, the spherical score and the ranked probability score — all scoring rules with the same property: inflating or deflating your confidence costs you.

The published training run: 7,313 updates, one epoch, about 1.96 hours. It looks like a typo and it is not — because the base already existed.

The raw model is worse than always answering the majority

The model card puts the raw checkpoint at 0.362. The majority baseline on the same evaluation — answer the most common label every time — is 0.461. Thinking about it at all makes things worse. The authors say so themselves: Laya is a fast base to specialise, not a zero-shot decision engine.

Confidence is the same story. Expected calibration error at ship is 0.466, which is barely a probability at all. Refitting the temperature brings it to 0.081 — and that work belongs to whoever adopts it.

What the adopter takes on

• Assemble thousands of labelled examples — by far the heaviest step
• Fine-tune with the vendor’s notebook — two free GPUs, about four hours
• Refit the temperature — one per question type and option count
• Design so that options stay under twenty; accuracy falls visibly beyond that
• Put a router in front to pick the checkpoint by language and script

In other words, what appeared in three days was a base, not a decision engine. Building the model did get lighter; the weight did not vanish, it moved to the data and the calibration.

The most instructive part of this episode: the same model’s numbers become different numbers depending on who published them.

The ECE ordering reverses depending on the source. So does accuracy, once a task has 77 options. The last note said Jev’s “193.6× faster” was vendor-run; the open-source side’s numbers are self-reported in exactly the same way.

“It is small, so run it locally” is tempting, but the criteria are more concrete than that. Cost first.

One worked estimate, assuming 800 tokens per decision, a single T4 in the Tokyo region and 0.2 FTE of operations, puts the break-even at 52.6 million decisions a month. Below that the API is cheaper. Routing support tickets will not get anywhere near it.

When local wins

• The data cannot leave — health, finance, government. This one is independent of volume
• It has to work offline — download the weights once and no connection is needed
• You want a lower latency floor — one network round trip disappears
• The same judgement repeats at scale — fraud checks on every transaction, continuous UGC moderation
• You already hold labelled data — the material for fine-tuning is there, which is what makes ownership worthwhile

How fast it actually runs on a laptop

The Core ML build on an M3 Max Neural Engine measures P50 of 4.98ms at 0.154 joules per decision. Against the MLX build on the same machine that is 1.39× the speed but 2.78× the energy efficiency. For continuous judgement on a battery-powered device, the second number matters more.

The Neural Engine build caps at 96 tokens in total, so the state has to be designed to fit. There is also an ONNX Runtime implementation callable from Node.js; the range of places to put this is widening.

So what to do

The fourth is the realistic one. Build it on the API and the logs become the labels for fine-tuning. The order makes sense on its own.