One gain, and everything that did not move

Nimbus 2 Apex follows instructions better than the model it was trained on, and that is the only claim on this page that survives a test of significance. IFEval rises 8.7 points: seventy-three questions are fixed against twenty-six broken, which chance would do less than once in a thousand times.

Everything else moved by less than the churn underneath it. That is worth saying plainly, because a five-point drop with ten questions fixed and seventeen broken looks like a result and is a coin toss. Every row below carries how often chance alone would move that many, and the ones that are noise are labelled noise.

The shipped adapter against Qwen3-8B, on the same questions.

Three arms we ran, and two figures we did not

The three solid columns were run by us, in one session, on one loaded model, with the adapter switched on, off or turned down — so the only difference between them is the adapter. The two hatched ones are their makers’ own published figures, and two of them are not even the same benchmark: each says underneath which one it is. Read them as the neighbourhood, not as a race. The scale is fixed from 0 to 100 for every chart, because a column drawn to fit its own chart makes four points look like forty.

Question by question

Fixed means the question was wrong without the adapter and right with it; broken is the reverse. Only the questions that changed their mind carry information, so the chance column is computed from those alone: if ten were fixed and seventeen broken, a coin does that about a quarter of the time.

The dial that decided which model ships

LoRA does not replace a model's weights; it adds a small update to them, and how hard it pushes is one number. At full strength the adapter wins IFEval by more and loses ten points of MMLU-Pro at level 5 — a real loss, not noise. Turned down to three quarters, the loss goes and most of the gain stays.

The reason is length, and it is measurable. On the same 140 questions the full-strength adapter writes fifteen per cent more reasoning than the base model, so the ladder's ceiling cuts it off mid-thought a third of the time instead of a fifth — and an answer that comes out of a severed reasoning is a worse answer. At three quarters the verbosity goes back to the base model's, and so does the truncation.

The ladder did not work

The model is built around a thinking level from 1 to 20, each with a word target. At level 20 the adapter writes 1,983 words of reasoning. The base model writes 1,979. Four words apart, after fourteen hours of training aimed at exactly this.

The cause is in the data, not the training. Each row's level was measured from the length of the reasoning it labelled — so the label was a function of the thing it was meant to predict. The model never saw one question answered at two different lengths, and could predict every row's text exactly as well by ignoring the level entirely. So it did. The label was descriptive where it needed to be instructive.

One thing did move. At level 5 the base model is cut off at the ceiling 86% of the time; the shipped adapter, 67%. It stops on its own a third of the time where the base model manages one time in seven. That is the ladder showing through once the verbosity is out of the way, and it is the thread the next round starts from.

The other model of ours, and the one number they share

Nimbus 1.1 Prime-EE came first, by six days: an adapter over Qwen2.5-Coder-7B-Instruct, trained out of the habits a good programmer has anyway — a hard-coded dir="ltr", a form control with no label, a container running as root. It was measured on HumanEval and MBPP, which are code benchmarks, and on a hand-built suite of 105 tasks scored before and after. Apex is measured on knowledge, mathematics, instruction-following and competitive programming. The two lists barely overlap because the two models were built to do different things.

IFEval is the one they both have, and it is worth reading carefully. The gap is large and most of it is not ours: the two adapters sit on bases a year apart. Qwen2.5-Coder-7B is a code model from 2024 and follows instructions worse than Qwen3-8B before either adapter is applied. What each adapter contributed is the distance from its own base — and only one of them has that measured. Apex’s is +8.7 points. For 1.1 the base was never run on IFEval, so its own contribution to 52.3 is unknown and this page will not guess at it.

One thing 1.1 has that Apex does not: an item-by-item before-and-after on a suite built for the job it was trained for, where its real claim lived — frontend tasks 9 out of 20 to 15 out of 20 — and where its one regression was stated before anyone asked, image specifications 18 down to 15. Apex has p-values instead. Both pages try to answer the same question, and the second one has a better ruler.

The Nimbus 1.1 Prime-EE page →

It reads pictures without being able to see

The weights are text-only and stay that way. A picture is turned into code before the model meets it, by a module that runs in front of it: the colours are reduced, the connected regions are found and named, and the words in the picture are read where they sit. What arrives is an XML scene — shapes with positions, sizes and colours, and text at its own coordinates — which the model reads the way it reads any other document. A bar chart even carries what each bar is worth, read off the chart’s own axis rather than guessed from its height.

<image module="apex-imagexo-v1" width="640" height="420" background="#fefefe">  <text x="317" y="29">Bugs closed per team</text>  <axis orientation="vertical" top="100" bottom="0"/>  <rect x="74" y="123" width="86" height="237" fill="#264653" value="86" label="Mar"/>  <rect x="187" y="307" width="86" height="53" fill="#2a9d8f" value="25" label="Apr"/></image>

It was never trained on one. The file of vision rows was started and never finished, so the adapter has seen exactly zero of these scenes — which makes the measurement an honest one rather than a rehearsal. On thirty-four questions about pictures it had never met in any form, it answered seventeen: tables nine out of fourteen, and maths problems printed inside the picture eight out of twenty. The base model answered sixteen. The ability is the base model’s; the module is what carries the picture to where that ability can reach it.

Charts, documents and diagrams were not in that sample and stay unmeasured until they are. Precision is not what the module is for: it is a reader of structure, and it says so — edges are approximate, two near colours become one, and a shape it cannot name is reported as a region rather than guessed at.

What Qwen publish for the same weights

Qwen report AIME 2024 at 76.0 for Qwen3-8B in thinking mode. We measured the same weights at 20.4. Both are true. Their harness lets the model think for sixteen thousand tokens; level 20 of our ladder allows about four and a half thousand and closes the reasoning itself when that runs out. The ceiling is what holds the number down, not the weights — and both of our arms run under the same ceiling, which is why the comparison above is still fair.

How it was trained