Between one bit and two.

The gap in the ladder

Class 03's residual codec stores a token as a centroid id plus a correction: the residual — what is left of the token after subtracting its nearest centroid — is rounded to a few bits per dimension, and nbits (1, 2 or 4) says how many. The rung you pick is most of the index's size. Drag the embedding dimension and look at the step between 1-bit and 2-bit:

Storage is the dominant cost of a late-interaction index — class 03 opened with that, and it does not get less true here. So the gap matters: between the cheapest rung that still carries a trained correction (1-bit) and the first one most people are willing to ship (2-bit) there is a factor of two in the biggest line item, and no way to ask for anything in between.

The constraint is only that nbits is a whole number. A symbol with three possible values carries log₂3 = 1.585 bits of information — that is simply how many bits it takes, on average, to tell three things apart. Three values per dimension would sit right in the gap. The only question is how to store 1.585-bit symbols in memory that is addressed in whole bytes, without wasting the fraction.

Five trits in a byte

A bit is a digit with two values. A trit is a digit with three — 0, 1 or 2 — and a codec that stores one trit per dimension is called ternary. Eight bits fill a byte exactly; trits don't divide a byte evenly, so you pack them the way you would write a number in base 3. Five trits make a five-digit base-3 number, and the largest such number is 242 — which fits in one byte because 3⁵ = 243 ≤ 256. Click the digits to build a byte:

Why five, and not some other group size? Packing k trits into whole bytes wastes whatever part of the last byte the 3ᵏ patterns don't fill, and that waste decides the bits per dimension you actually pay:

Four trits use only 81 of a byte's 256 patterns and land at exactly 2.0 bits per dimension — the storage win is gone; you have re-derived the 2-bit codec with extra steps. Six trits need 729 patterns, spill into two bytes, and land at 2.67, worse than 2-bit. Ten trits in two bytes tie five-in-one at 99.1% but buy nothing and double the group you must decode at once. Five is the smallest group that reaches the ceiling.

In code, the encoder asks each residual value which of the three buckets it falls in — searchsorted against the two cutoffs (chapter 04 is about where those cutoffs go) — then takes a dot product with the place values. The decoder is a 256-row table. Both are four lines of nanoplaid.py:

# encode: trits -> one byte per 5 dims, place values 3^0..3^4
trits = np.searchsorted(codec.cutoffs, residuals)        # 0, 1, 2
packed = trits.reshape(n, -1, 5) @ _TRIT_PLACE        # [1,3,9,27,81]

# decode: one indexed load resolves five dimensions
codes = _TRIT_LUT[packed].reshape(n, -1)[:, :dim]
recon = codec.weights[codes]

Notice what the decoder doesn't do. The scalar codec unpacks per dimension — shift, mask, index a 2^nbits table, 128 times per token at dim 128. Ternary does 26 table lookups per token, each resolving five dimensions at once. That asymmetry is the whole of the next chapter.

The kernel doesn't care what base you packed in — until it unpacks

Class 05's fused identity is why this rung was cheap to add. A token decodes to centroid[cid] + weights[codes], so its dot product with a query row splits into a centroid term stage 1 already computed, plus an integer dot product between the int8 query and the int8 bucket table — the short list of values a code can decode to. Nothing in that derivation mentions how many bits a code has, or how the codes were packed:

So ternary inherits the kernel for free. In nanoplaid.py the whole integration is one branch inside unpack_codes, and the self-test checks that the fused score matches a literal per-element integer loop for base 3 exactly as it does for base 2 — same integers, same float32 operation order. The bucket table has 3 entries instead of 2, 4 or 16; nothing else changes.

for scheme, nbits in (("residual", 4), ("residual", 2), ("ternary", 0), ("residual", 1)):
    ...
    # the fused score must match a literal per-element integer loop
    assert slow == fast[i, j], f"LUT integer spec drifted ({scheme})"
    # ...and stay within int8 rounding of the float decode path
    assert err < 0.01

Now the one place the packing is not free — and it is the place this whole school is about. Classes 04 and 05 do their table lookups inside a vector register, with NEON tbl or AVX2 pshufb. Those instructions look up a 16-entry table using a nibble — four bits, half a byte — as the index. That is exactly why base 2 is so comfortable: a 2-bit or 4-bit codec's byte splits into nibbles, each nibble indexes the shuffle, and decode never leaves the vector unit. A base-3 byte holds 243 patterns and does not split: there is no way to turn 243 into two 16-entry lookups. Any packing that does split costs at least 2 bits per dimension, which is the 2-bit codec with the storage win gone.

Given all that, ternary should be the slow rung. It is the fastest one:

How to read it: B/token is the residual payload plus the 4-byte centroid id. The numpy column is this repo — eval.py data/scifact, median (p50) milliseconds per query on the machine this class was written on. The two right-hand columns are next-plaid's Rust codecs on GitHub's CI runners — ubuntu-24.04-arm (Neoverse-N2) and ubuntu-latest (EPYC 7763, AVX2): decode throughput in millions of elements per second, and end-to-end query time relative to 2-bit. Every codec was timed in one process, because the same benchmark moves ±30% between runners.

The reason is chapter 02's. A scalar codec pays per dimension — shift, mask, index, 128 times per token at dim 128 — and 4-bit's shifts are the widest, which is why 4-bit is the slowest rung end to end despite being the most faithful. Ternary pays per byte: 26 indexed loads, each resolving five dimensions. Even against 1-bit, which reads 10 fewer bytes per token, the arithmetic saved outweighs the extra traffic.

Quote the end-to-end column, not the decode column. In isolation ternary decodes 1.33× (ARM) and 1.90× (x86) faster than 2-bit; inside a real query that shrinks to 1.09× and 1.41×, because decode overlaps with the MaxSim dot products it feeds. Class 06 made the same point about stage-1 phases: an isolated microbenchmark tells you the mechanism, not the claim.

That is the mechanics, complete: smaller than 2-bit, faster than 2-bit, and the kernel did not have to learn anything. Everything from here on is about quality — and quality turns out to hang on one number the packing never mentioned.

The dead zone is the setting, not a detail

Three buckets means two cutoffs, and the only shape decision is how wide the middle bucket is — the one that decodes to exactly zero. Call it the dead zone. Class 03's machinery offers a default: cut at equal mass, the 1/3 and 2/3 quantiles of the residual values, so each bucket gets a third of them. That zeroes a third of every dimension no matter what the residuals look like. It feels like the neutral choice. It is the codec's worst setting.

The alternative is to measure the residuals' spread and place the zone against it. Write σ for the standard deviation of the residual values — their typical size. A dimension is stored as 0 when |r| < τ·σ, and as ±m otherwise, where m is the average size of the values that survive. So τ — the dead zone's half-width, in units of σ — is the codec's one setting. Drag it and watch what the codec is actually trading:

Two things fall out of the picture. First, the equal-mass split is not "no choice". On roughly bell-shaped residuals it is silently choosing τ ≈ 0.43 — press the button and see. It looks neutral because it is written as a quantile, but it is a specific, and poor, dead-zone width, arrived at by copying the scalar path's formula with n_options = 3.

Second — and this one is worth carrying out of the class — reconstruction error and ranking quality disagree about where the best τ is. MSE is lowest near τ ≈ 0.61 and nearly symmetric around it, so it rates τ = 0.43 and τ = 0.80 as almost equally bad: 0.2067 against 0.2078, one part in two hundred apart. NDCG separates them decisively. In chapter 07's seven cells, τ = 0.43 loses to the τ-based dead zone in 7 of 7, while τ = 0.80 is roughly a wash against 2-bit. Two settings that reconstruction calls interchangeable are not interchangeable for ranking — and the asymmetry runs toward the wide side, where MSE cannot see it.

So the width matters, the free construction gets it wrong, and the cheap metric can't be trusted to find it. That leaves the question: where should the zone be? You could sweep τ against NDCG on every corpus you care about. Or you could notice that this is not an open question at all.

Solve for it

There is a textbook answer. Lloyd–Max is the pair of conditions any quantizer that minimizes squared error must satisfy: each decoded value is the average of the values in its bucket, and each cutoff sits halfway between two neighbouring decoded values. For three levels [−m, 0, +m] with a dead zone of half-width t, that is the whole system:

m = E[ |r| : |r| >= t ]     level = centroid of its cell
t = m / 2                   cutoff = midpoint of 0 and m

Two equations, two unknowns, and they are not independent: substitute one into the other and you have a single equation in t whose solution is a fixed point — a value the rule maps back onto itself. Alternate the two lines and they converge in a handful of steps from any starting guess. On bell-shaped (Gaussian) residuals the answer is τ = t/σ = 0.612 — a constant you can derive, not one you fit — and it sits just below the best-MSE point in chapter 04's widget, because minimizing MSE is exactly what these two conditions encode.

So there is nothing to tune. Whatever τ your corpus wants, lloyd_tau() computes it from the residual sample you already drew to train the codec — a dozen lines, no evaluation loop, no held-out set, no queries. The interesting question is what makes a corpus want something other than 0.612, and the widget below answers it. Every token in it has a perfectly Gaussian direction. The slider changes only how much the residuals' lengths vary from token to token — the norm CV, the spread of residual norms divided by their mean:

Nothing in this widget is heavy-tailed by construction. Yet as soon as the norms vary, the pooled distribution grows heavier tails — its excess kurtosis rises — and τ moves out to cover them. The mechanism is forced. Write a residual as a length times a direction, r = s·u. Fourth moments factor, so

3 + kurt(r) = ( 3 + kurt(u) ) · E[s⁴]/E[s²]²

PLAID trains one quantizer over every token in the corpus. A token near its centroid and a token far from it go into the same histogram. Pooling values that share a shape but differ in scale is called a scale mixture, and a scale mixture is heavy-tailed by construction: the second factor is greater than 1 whenever s varies at all (Jensen's inequality — the average of a square exceeds the square of the average). Heavy tails come out of the pooling whether or not anything about the encoder is heavy-tailed, and τ moves out to cover them.

And the widget is not just an illustration. Set the slider to a real cell's measured norm spread and it returns that cell's measured τ — synthetic tokens, nothing matched but one number:

Four cells, four agreements inside 0.004. Whatever else is going on inside a ColBERT checkpoint, the number this codec turns on is reproduced by a lognormal spread of norms over Gaussian directions, and it takes exactly one parameter to do it.

τ is a kurtosis readout

If that is the mechanism, the solved τ should be predictable from the residual shape alone — no retrieval, no queries, no relevance labels. Sixteen cells (an encoder × corpus pair is a cell), five k-means seeds each, τ solved on the same residual sample the codec trains on:

The rank correlation — Spearman ρ, which asks only whether higher kurtosis goes with higher τ — is 0.93 across all sixteen and 0.99 within the twelve dim-128 cells. What scatter remains is dimension: the two dim-48 cells sit above the trend and the two dim-96 cells below it, which is why the within-dimension correlation is the tighter one. Nothing here touched a relevance label.

You can reproduce the shape of it on a laptop. The same command runs on the committed toy in about a minute, or on any bundle you have encoded — these seven are full corpora, one k-means each, no retrieval run at all:

python eval.py data/toy --residual-stats        # the 1-minute version
python eval.py data/<bundle> --residual-stats   # the table below

Two columns carry the story. mix — the part of the heavy tail that comes from norms varying, the E[s⁴]/E[s²]² factor from chapter 05 — accounts for essentially all of kurt in every row. And dir, the tail heaviness of the residual directions alone, already sits within a few thousandths of −6/(dim+2): the value a single coordinate takes when directions are spread evenly over the sphere, which this class calls the sphere floor. Hold onto that number; chapter 10 is where it earns its keep.

What the wrong τ costs

Changing τ changes nothing about the format — no extra byte, identical decode speed, the payload is bit-for-bit the same size at every τ. Only the three values in the bucket table move. So the quality cost of a bad τ comes down to one number: the NDCG@10 difference against 2-bit on the same corpus. Seven cells, each built on one k-means run shared by every column, so centroids, residuals and queries are identical and every difference is paired:

How to read it: q is the number of judged queries; 2-bit is that codec's absolute NDCG@10; every other column is the ternary setting's NDCG@10 minus 2-bit's, green when ternary is ahead. equal-mass is chapter 04's default split; the three τ columns are dead zones placed by σ. The last two rows pool the seven cells and count how many came out positive. Command: python eval.py data/<bundle> --schemes residual2,ternary@mass,ternary@0.50,ternary,ternary@0.80. Every cell here passes chapter 12's noise gate.

Read the equal-mass column first, because it is the only thing this table settles. Against the τ-based dead zone it loses in 7 of 7 cells, by −0.0024 on average and −0.0070 in one cell. The free construction — the scalar path's quantile formula with three buckets — is reliably the worst way to place the zero bucket. That is the claim worth carrying, and you can reproduce it on a laptop in about twenty minutes.

Now read across a row, and notice that the table does not settle which positive τ is best. The four settings differ by up to 0.0105 within a single cell — three times the effect being measured — and the cells disagree about the direction: SciFact ranks τ = 0.50 first and τ = 0.80 last; POJ-104 on the code encoder ranks τ = 0.80 first by a wide margin. Pooled over all seven, τ = 0.50 and τ = 0.65 tie at +0.0008 over 2-bit and τ = 0.80 trails at +0.0004. That is not an answer.

Settling the fine question takes more queries, more seeds and a second implementation. The first pass at it is next-plaid#177: 8 cells, 4,240 judged queries, Rust. There τ = 0.65 beats 2-bit by +0.0028, positive in 8 of 8; equal-mass loses by 0.0029; and τ = 0.80 is close enough (+0.0027, and ahead head-to-head by +0.0005) that 0.65 ships for never losing rather than for winning on average. Note which parts agree with the table above and which don't: the equal-mass penalty reproduces across implementations; the margin over 2-bit is 3× smaller here, and the τ ordering doesn't reproduce at all. Nothing is broken — that is what a 0.003 effect looks like when you measure it seven times. The next two chapters are the second pass, with five seeds per cell and the decomposition the first pass couldn't afford.

Criterion against rate

"Ternary matches 2-bit at 19% fewer bytes" is an outcome, not an explanation. The codec spends 1.585 bits where 2-bit spends 2, so something has to pay the difference back, and there are only two candidates: the number of levels — the rate — or where the levels are placed — the criterion. Six cells, one k-means per cell shared across all arms (an arm is one codec configuration in a comparison), with both reconstruction error and NDCG measured on each:

The first row settles the accounting. At an identical 1.585-bit rate — same levels, same bytes, same decode — placing the zero bucket by σ instead of at the equal-mass quantiles cuts reconstruction error by 18 to 26 percent, in 6 cells of 6. The criterion buys the bits back: ternary's effective fidelity is already 2-bit's, which is why it can stand next to 2-bit at 26 bytes per token.

The second row is the one to stare at. Against real 2-bit — 26% more bits — reconstruction error is a wash: the ratios run 0.93 to 1.12, straddling 1, ternary ahead in four cells and behind in two. Equal fidelity, by the metric that measures fidelity. And NDCG is +0.0045, positive in 6 of 6.

The third row is where it runs out. Against 4-bit — 2.5× the payload bytes, and 2.3 to 5× lower reconstruction error — ternary gives back 0.0005, and the sign splits 2 of 6. Fidelity keeps improving and ranking stops caring. That saturation is what this whole rung depends on: ternary is not a cheap approximation of 4-bit, it sits at the point where the ladder has already flattened. Chapter 11 finds the corpus where that stops being true.

Which decision carries it

Chapters 05 and 06 showed how to solve for τ and that the solution tracks a real property of the checkpoint; chapter 07 left "which τ" open. The obvious move is to ship the solved τ instead of the constant 0.65. We measured that expecting a small win, and got nothing — and the shape of the nothing is more useful than the win would have been.

There are three separable decisions hiding inside "use a dead zone", and they are usually discussed as one. Eleven cells, five k-means seeds each, every contrast paired within a seed:

Having a zero level at all is the entire effect: +0.0069, positive in 11 cells of 11. Lloyd placement without a zero level — solving for four levels instead of three, the same optimality argument with no dead zone in it — gets a quarter of that, in 8 of 11. And solving for the dead-zone width instead of assuming 0.65 is worth +0.0004, indistinguishable from zero at this resolution.

That last one deserves a real bound rather than a shrug, because "we didn't find a difference" is not the same claim as "there isn't one". So it got its own experiment: 13 cells × 5 seeds × 3 arms, regressing the per-query paired difference on how far the solved τ sits from the shipped 0.65 — its displacement. If solving helps, cells where the two disagree more should gain more:

The slope is +0.011, with a confidence interval straddling zero and no individual cell significant. Multiply it through by how far τ actually moves. Across the 13 cells the displacement runs 0.010 to 0.049, median 0.014; evaluate at the top of that range — the most generous case available — and the point estimate implies +0.0005, with the interval's optimistic edge at +0.0016. At the median displacement it is +0.0002. This is not an underpowered shrug. It is a null with a ceiling on it, and the ceiling is below anything you would ship for.

The mirror arm is what makes it a measurement instead of an absence. It displaces τ by the same distance the wrong way (0.65 − Δ instead of 0.65 + Δ), so a real optimality effect has to show up there at roughly double the gap. It reads +0.022 against +0.011 — a ratio of 2.0, exactly as predicted — with a confidence interval that also straddles zero. Both arms are consistent with the effect being real and tiny, and with it being nothing. What neither is consistent with is an effect worth a migration.

Rotate it

Every codebook-free quantizer of the last few years — RaBitQ, TurboQuant and their descendants — rotates first: it multiplies every vector by one random orthogonal matrix, a rotation, which preserves all lengths and dot products. A random rotation mixes each coordinate with all the others, so every coordinate's distribution comes out bell-shaped — it Gaussianizes — and that is exactly what licenses a precomputed, data-independent codebook. So a codec whose story is "τ tracks the heavy tail" invites one obvious objection: rotate the residuals, the heavy tail goes away, and with it your effect.

The experiment's design is forced by that same length-preservation. Apply one rotation R to documents, queries and the centroid table together, and every stage before the codec is untouched:

MaxSim:      (qR)(dR)ᵀ = q dᵀ                    float baseline unchanged
assignment:  argmax (dR)(CR)ᵀ = argmax d Cᵀ      centroid codes unchanged
residual:    (dR) − (cR)[code] = (d − c[code])R  residual exactly rotated

The centroid codes come out bit-identical, so rotation is the only difference between the two runs — no re-clustering noise. And the float row becomes a free check on the whole path: if it moves, the plumbing is broken and nothing else in the run may be read. Four cells, five seeds, a Haar-random R — uniformly random over all rotations — at a seed declared in advance:

The heavy tail does not go away. Pooled kurtosis moves by at most 0.02 in any cell, the solved τ by at most 0.0005, and the criterion gain survives in 5 seeds of 5 everywhere. Chapter 06's spare fact explains why in one line: the direction kurtosis was already at the sphere floor −6/(dim+2) before the rotation. There was nothing for R to Gaussianize. Every bit of the heavy tail was the norm mixture — and since a rotation preserves lengths,

‖rR‖ = ‖r‖

a rotation cannot touch a scale mixture, ever, for any R. The literature's Gaussianization argument has a precondition that is easy to miss, because single-vector methods satisfy it for free: they quantize unit-normalized vectors, where there is no norm mixture to begin with. Late interaction cannot do that, because the residual norm carries MaxSim signal — normalize it away and you are scoring a different quantity. The floor is not an artifact of our pipeline; it is structural to quantizing un-normalized residuals.

One more thing the control ruled out. At 1 bit, in an earlier arm of the same program, rotation was catastrophic: 0.327 → 0.129 at equal bytes and equal reconstruction error — a 61% collapse in ranking, driven by the rotation spreading quantization error into the query's direction on a corpus where the useful signal is under half a percent of the mean score. Whether that mechanism reached up to 1.585 bits was unmeasured, and it is the kind of thing that would quietly poison a rotated arm. It does not: the worst cost any arm paid here is 0.0046, and the ternary arms stay inside 0.0025.

Where it stops

If the effect is manufactured by the spread of residual norms, then a corpus whose residual norms don't spread should not show it. That is a falsifiable prediction with an obvious test case, and it is the one place in this class where the codec loses.

Long documents. MLDR — a long-document retrieval benchmark — in English: 8k documents averaging 3,267 tokens each, 800 queries. That is an order of magnitude more tokens per document than anything else in this class, so k-means has far more material per cluster and residuals come out tight:

How to read it: the middle column is the range across the NFCorpus cells of chapter 13's grid; the right column is the long-document corpus. The criterion gain is dead-zone ternary minus equal-mass ternary; the last row is 4-bit minus dead-zone ternary, so a positive number means 4-bit wins. In brackets, how many of the cells — or, for MLDR, of its three seeds — agreed on the direction.

Kurtosis falls to +0.78 and τ lands at 0.624 — within 0.012 of the Gaussian fixed point, the closest any cell in the program gets. The criterion gain shrinks by a factor of four, and 4-bit pulls ahead of ternary instead of tying it. Every quantity moves in the direction the mechanism predicts, together, on a corpus chosen for the property rather than for the outcome.

So the honest scope of the quality claim is: ternary ≈ 4-bit is conditional on heavy tails, and heavy tails come from the residual-norm spread. Where they are absent, the ladder reverts to rate — more bits, better ranking — and you are back to choosing on bytes alone.

That is not a retraction, because the footprint argument survives intact: ternary still holds 99.6% of float NDCG at 26 bytes per token on those 3,267-token documents. It stops being free, not useful. And the tell is free too: --residual-stats reports the kurtosis before you encode anything, and a cell reading below about +1 is one where you should expect the dead zone to be worth little, and check 4-bit.

Reading three thousandths honestly

Everything from chapter 07 on is a difference of a few thousandths of NDCG. At that size the real risk is not choosing wrong — it's measuring wrong and never noticing. This chapter is the checks that made those numbers readable, starting with one that costs nothing because the row is already in the table.

1-bit is strictly lossier than float. It throws away more than any other rung, so on a corpus where the ranking is well determined it cannot beat the exhaustive float baseline — exact MaxSim in float32, the reference every codec is measured against. Any cell that reports 1-bit above float is reporting noise, and you get that verdict free, from a row you printed anyway. Here is this repo's own committed toy, which fails the check in two of four domains:

The gate catches two different failures, and only one of them is fixable with money. Too few judged queries — queries that come with relevance labels — is the toy above, and it's the cheap case: a query costs one forward pass and never touches the corpus encode, so buying more is the least expensive thing in retrieval evaluation. In the next-plaid study, taking one cell from 130 judged queries to 1,000 flipped the sign of the effect being measured, from −0.0008 to +0.0027. Not tightened the interval: flipped it. Too low a float ceiling is not fixable at any budget. A code-specialized encoder run over financial prose failed this gate with 648 judged queries and all seven lossy profiles above float, because at a float NDCG@10 of 0.2492 the ranking is too weakly determined for quantization noise to push it in only one direction. Judged queries are necessary, not sufficient — screen a candidate cell on its float ceiling before paying to encode it.

One more from the same study: the cell that failed the gate hardest had a perfectly well-ordered reconCos column — reconstruction cosine, a fidelity score like MSE. Reconstruction fidelity stayed monotone in bytes while NDCG was pure noise. Robust and wrong — chapter 04's lesson, arriving from the opposite direction.

The gate is for cells that can't answer. The rest of this chapter is how the cells that could answer were made to — four ordinary choices, each one here because skipping it produced a wrong answer first.

One k-means per cell, shared by every arm. Every contrast from chapter 07 onward is a difference between two codecs built on the same centroids, the same codes, the same residuals. Only the bucket table differs. That removes clustering variance from the difference entirely — which matters, because clustering variance is larger than every effect here.

Five seeds, and report the sign count. Every GPU-study number in this class carries how many seeds agreed on the direction, next to the mean, because at this effect size those say different things. A +0.005 that is 5/5 and a +0.005 that is 3/5 look identical in a mean-only table and are not the same finding. The convention — the mean, its spread across seeds (the seed SD), the sign count, and whether the mean clears its own SD — is four numbers instead of one, and none of them is optional.

A control that can fail in a known direction. Chapter 09's mirror arm and chapter 10's float-baseline gate are the same idea: build into the run something whose value you can predict if your hypothesis is right, and which is free. The mirror arm predicts 2× the gap; the rotated float row predicts an exact match. Both were checked before any codec number was read out of those runs.

Predictions written down before the run. Chapter 10's seven predictions — including the one that would have killed the story — were committed to the repository before a single rotated cell existed. It is the cheapest possible discipline, and the only defence against the version of yourself who, having seen the numbers, remembers expecting them.

The scoreboard

Two tables, two scales. First the one you can run yourself: full SciFact, every scheme this repo implements, one k-means shared across all of them — python eval.py data/scifact, on the machine this class was written on:

How to read it: build s is the time to train the codec and encode the index; B/token is everything stored per token, centroid id included; retention is NDCG@10 as a fraction of the exhaustive float row; p50 ms is the median query time.

Ternary sits at 30 bytes per token — 26 of payload plus the 4-byte centroid id — with the float ceiling's NDCG intact, between residual-2's 36 and residual-1's 20, and it is the fastest numpy rescore of the three. Read the retention column with chapter 12 in mind: SciFact's 300 queries resolve the big steps — residual-1 and binary losing about 2 points is real and reproducible — but residual-2 reading 100.1% of exact is the resolution floor showing through, not a codec that improved on the float it approximates. This table shows ternary and 2-bit are in the same class; it takes the second table to separate them.

Five encoders on one corpus, five k-means seeds each, every arm built on the shared codebook of its seed. NDCG@10, and retention against that cell's exhaustive float baseline. Dimension varies on purpose — it is the second-largest source of variation in this class after the checkpoint:

How to read it: one block per encoder, with its dimension, pooled kurtosis and exact-float NDCG@10 under its name; five arms per block, from 1-bit up to 4-bit. payload B/token is residual bytes only. In each block, compare the green ternary row with the 2-bit row just below it and the 4-bit row at the bottom.

Against 2-bit, ternary is ahead in all five blocks, spending 19% fewer payload bytes at dim 128 — 17% at dims 96 and 48, where the five-trit byte has to pad — and across the wider 11-cell grid it is ahead in 10. Against 4-bit it gives back 0.0019 on average for 41% of the bytes. The mxbai block is the one to look at twice: at dim 48 there are only 48 residual coordinates to spend anything on, every codec is further from float, and the spread between codecs is three to four times wider than at dim 128 — the criterion gain reaches +0.0136 and 1-bit falls to 0.859 retention. Small dimensions do not make quantization choices matter less. They make them matter more.

Where this leaves the ladder: binary when you can afford ~2 points and want the fastest possible stage 2 (class 04's kernel is unbeatable — 17.9 ms, at the float baseline's speed); ternary when you're footprint-bound and want the quality back, and the residual kurtosis reads above about +1; residual-2 when you'd rather spend 6 more bytes per token than think about it; residual-4 when --residual-stats says the tail is light — long documents, tight clusters — and essentially never otherwise, on this evidence: it costs 2.3× the bytes of ternary and 1.8× the time to be no more accurate. And whatever you pick, place the dead zone by σ, not by quantile, and leave τ at 0.65: the one setting that matters has a solvable answer, the answer explains the codec, and the constant already sits on it.

Check yourself

Instant feedback, explanations included. Miss one? The chapter number is next to it.