Pooling-Aware Training Keeps Multi-Vector Search Accurate With Fewer Vectors

At a glance

Multi-vector embedding models match queries precisely by making a vector for each token, but the many vectors take up a lot of storage. This article explains how to train a model specifically for token pooling, which merges similar vectors, so that useful information survives the merge.

Key points

1. 1

   Multi-vector models can create hundreds of vectors per document, so storage grows quickly across millions of documents.

2. 2

   Token pooling merges similar vectors to reduce their number, but stronger compression makes it easier to lose useful information.

3. 3

   With pool_factor=4, the author says a normally trained model kept 94.6% of its unpooled search quality, which rose to 97.8% with pooling-aware training.

4. 4

   The method groups nearby vectors with Ward hierarchical clustering and represents each group by its average vector.

Why it matters

Search services can get better results from the same storage budget, cutting costs while keeping quality.