Cutting an ordinary embedding vector in half destroys it. The information is distributed across all dimensions with no ordering, so the first half is not a smaller version of the whole — it is an arbitrary subset that has lost most of the signal.
Matryoshka representation learning changes that by training the model so that prefixes of the vector are themselves useful embeddings. The first quarter works, the first half works better, the whole thing works best. Each nested prefix is a coherent representation at its own size.
What changed in 2026
- It became a standard feature of new embedding models. Rather than a research technique, matryoshka training appeared as an advertised capability in mainstream embedding model releases.
- Two-stage retrieval spread. Searching a truncated index and refining with full vectors became a common architecture for large corpora.
- Storage economics drove adoption. As embedding indexes grew into serious infrastructure costs, the ability to halve dimensions with modest quality loss became directly valuable.
- Combination with quantization matured. Truncating dimensions and reducing precision together produced compounding savings.
What truncation buys
| Dimension retained |
Typical relative quality |
Storage |
| Full |
Baseline |
Baseline |
| Half |
Close to baseline for many corpora |
Half |
| Quarter |
Noticeably lower, often still usable |
Quarter |
| Eighth |
Coarse; useful as a first-stage filter |
An eighth |
The exact curve depends heavily on your corpus and task, which is why measuring rather than assuming matters. Some corpora tolerate aggressive truncation with almost no observable recall loss; others degrade quickly. A single experiment on a labelled set settles it.
The two-stage pattern is where the technique earns most. Index truncated vectors, search them to retrieve a generous candidate set cheaply, then rescore those candidates with full-dimension vectors. Most of the search cost is paid at the short dimension, and final ordering uses the full representation — quality close to full-vector retrieval at a fraction of the search cost.
Practical considerations
Truncate consistently. Every vector compared must be the same length, so the index and the query embedding must match. Mixing lengths produces meaningless similarities rather than an error, which makes it a silent failure worth guarding against in code.
Normalize after truncating. A truncated vector generally needs renormalizing before cosine similarity behaves as expected, and skipping this produces subtly wrong rankings.
Compose with quantization. Reducing dimensions and reducing per-dimension precision are independent levers, and applying both compounds the storage saving. Measure the combination rather than each separately, since the quality effects interact.
Verify your model supports it. Truncating an embedding from a model not trained this way does not work, and there is no error to tell you — retrieval quality just collapses. Check the model documentation before assuming.
For the broader question of when embeddings need regenerating entirely, embedding drift explained covers the maintenance side.
Common mistakes
- Truncating a model not trained for it. Silent quality collapse.
- Mixing dimension lengths in one index. Meaningless comparisons, no error raised.
- Not renormalizing after truncation. Subtly wrong similarity scores.
- Choosing a dimension without measuring. The curve is corpus-specific.
- Ignoring the two-stage option. The largest practical benefit and frequently unused.
FAQ
How much quality do I lose at half dimensions?
Frequently very little, and it depends on the corpus and model. Measure recall at a few dimensions on your own labelled set.
Does this work with any vector database?
Yes — truncated vectors are just shorter vectors. The database does not need special support.
Can I change dimension later?
Only by re-embedding the corpus, since the index must be consistent. Choose deliberately.
Does it help query latency or just storage?
Both. Shorter vectors mean less computation per comparison as well as less storage.
Where to go next
For alternative representations, read sparse embeddings explained and late interaction retrieval. For index maintenance, embedding drift explained.