TurboQuant quantizes a vector x one coordinate at a time, so it would like every coordinate to have the same, known distribution and to be independent of the others. Real vectors (for example keys in a KV cache) are nothing like that: a few “outlier channels” are much larger than the rest. The fix is to store ‖x‖ separately and multiply the unit vector x/‖x‖ by a random rotation Π (an orthogonal matrix, so lengths and inner products are unchanged and ΠT undoes it).
Πx is then a uniformly random point on the unit sphere, so each coordinate t = (Πx)j has density fd(t) ∝ (1 − t2)(d−3)/2 on [−1, 1]: a Beta distribution, ((t+1)/2 ~ Beta((d−1)/2, (d−1)/2)), with variance 1/d, which approaches the Gaussian N(0, 1/d) as d grows. This holds whatever x was.
Pick a unit vector x and draw many rotations. The histogram collects coordinate i of Πx over all the rotations drawn; the scatter plots coordinate i against coordinate j.
Things to try: the histogram is the same for every choice of x. At d = 2 the density is U-shaped and the scatter is a circle: the two coordinates are uncorrelated but completely dependent (t12 + t22 = 1). At d = 3 the density is flat (Archimedes' hat-box theorem). As d grows the scatter becomes a round Gaussian blob: the constraint Σtj2 = 1 is spread over many coordinates, so any two are nearly independent.
TurboQuant draws Π once and applies it to every vector. Below, N = 256 unit vectors before and after a single fixed rotation. The heat maps show the first 48 vectors (rows) by coordinate (columns).
The middle charts quantize every coordinate with the b-bit Lloyd–Max codebook designed for fd (next page) and show each vector's squared error ‖x − Q(x)‖2. Before rotating, the outlier coordinates fall far outside the codebook's range and dominate the error; after rotating, every vector's coordinates look like draws from fd, and the error is close to the value the codebook was designed for (dashed line).
Note what a fixed rotation does not do: it does not decorrelate the coordinates across a dataset. The covariance of the rotated data is ΠΣΠT, the same matrix in a different basis, so correlation between coordinates can go up as well as down (compare the numbers above, and try the low-rank data, where all the vectors share a 2-D subspace). Vectors that resemble each other still resemble each other after rotating, so columns of the heat map can still show a pattern. The guarantee is per vector, over the random choice of Π, and that is all TurboQuant's error bounds need.