Vector database: similar meaning, wrong document?

Intermediate vector database design: choose HNSW or IVF, enforce access filters, synchronize embeddings, and measure search quality beyond latency.

14 phút đọcTrung cấpEyeBlogs

Đọc bản tiếng Việt →

An original illustration: the question When will my money be back points to two Refund policy folders. Your company’s folder is labeled Access allowed; another company’s folder has a lock and is labeled No access. The headline reads Similar meaning ≠ right document.
Mục lục

An employee asks, “After a customer returns an item, when will their money be back?” The search box understands that “money back” is close in meaning to “refund.” But the top document belongs to another company, while the right instructions never make the list. A vector database can retrieve the wrong document even when it computes distances correctly. Similar meaning does not guarantee the right customer, the current version, or permission to read it.

Designing this system means bringing three things together: representing content as numbers, finding nearby representations, and managing data and access conditions. This article follows a hypothetical enterprise document collection. The question throughout is: how do we return semantically similar passages from the right set of authorized documents, even as the data changes?

How is a vector database different from a search library?

Imagine a librarian hearing “When will my money be back?” and finding a policy titled “Refund processing times.” The wording differs, but the subject is the same. An embedding is a sequence of numbers produced by a model to represent content, like coordinates on a map of topics. Related passages are expected to lie near one another under the chosen metric. That is not a guarantee that both passages are correct.

The article on image search explains the intuition behind coordinates and nearest neighbors. Here, the difficult work begins after we have the coordinates. A library such as FAISS can help find nearby points. A production service must also handle durability, updates, filtering, recovery, and query distribution.

A vector database supports storing and querying vectors alongside associated information. Think of a library with both a map for finding books and a catalog recording which books are current, which branch owns them, and who may read them. It can be a dedicated system or a familiar database with vector capabilities, such as PostgreSQL with pgvector[1].

The database does not necessarily create the embeddings itself. The model may run in another service. Changing storage systems cannot fix a poor representation model. Making search faster cannot turn an outdated document into the current policy, either.

Before choosing an index, what must the search box promise?

Consider an illustrative workload: 10 million document chunks belonging to multiple companies. Each query needs up to 10 results, drawn only from the signed-in user’s company and documents they may read. New content may take a little time to appear. Revoking access must not expose content during that delay.

The design team sets hypothetical targets: 95% of queries should finish within 300 milliseconds, and content changes should become searchable within 60 seconds. These are requirements for discussing the design, not measurements of a product. “95% within 300 milliseconds” means that, after sorting query times from fastest to slowest, the 95th-percentile value must stay within that limit.

Three promises need to stay separate. Correct scope means never returning unauthorized documents, just as a librarian should not hand personnel files to someone without permission. Correct neighbors means finding the nearest vectors within that scope. Usefulness means that the retrieved passages actually answer the question. Two passages with similar wording can describe policies from different years.

The read path therefore does not end with a vector search call. The authentication service establishes the company and permissions on the server side. The question becomes a vector using a configuration compatible with the collection. The system retrieves candidates within the permitted scope, ranks them, checks current versions and permissions, and only then reads the content to return.

If those passages supply context for a chatbot, the checks must happen before passing them to the answer-generating model. Filtering the final response cannot take back data already sent to another service. The company filter must not simply trust a company identifier supplied by the user, either.

How many sets of coordinates should one document have?

A long refund procedure may cover return eligibility, processing times, and exceptions for different payment methods. Compressing the entire document into one vector is like squeezing a whole handbook onto a single catalog card: the main topic survives, but small details can disappear. Systems therefore often split documents into retrieval chunks, like marking individual handbook sections so they can be looked up separately.

A chunk that is too short loses the conditions around a statement. One that is too long mixes topics. Splitting at headings or paragraphs is a reasonable starting point, followed by evaluation with real questions. Overlap between chunks preserves context across a boundary, but creates near-duplicate results. When returning results, group by document or deduplicate so the user does not receive ten near-copies of one passage.

Each record needs enough information to trace it back to its source and recognize stale data:

Information Illustrative example Purpose
Identity Company A, document 42, chunk 3 Avoid mixing chunks across customers
Content version Version 7, content checksum Avoid using a version 6 embedding for version 7 text
Representation Embedding space E2, chunking scheme C3 Identify the configuration that produced the vector
Access conditions Authorized groups, active status Filter results and check permissions
Content Text or a pointer to the correct source version Display, cite, and read the source

An embedding space is like a map’s coordinate system. Two models returning sequences of 768 numbers do not necessarily use the same coordinates. Record the model version, text preprocessing, query and document embedding conventions, and distance metric. Some models require different inputs for queries and documents; their paired configurations must remain compatible.

Cosine compares vector directions, like checking how closely two arrows point the same way. Euclidean distance measures the straight line between two points. Choose a metric based on the model and evaluation, not the database default. Qdrant supports multiple named vectors on one point, each with its own dimensions and distance metric. This lets representations coexist separately; it does not automatically make them comparable[3].

Why does fetching 10 and then filtering fail to return 10?

You ask the librarian for the ten books closest to your topic across the entire library. Only afterward do you remove books from reading rooms you cannot enter. All ten might be removed. Meanwhile, suitable books still sit on shelves you may access, but nobody has fetched them.

This is the failure mode of post-filtering: retrieve a candidate set, then apply conditions. The figure below deliberately uses exact distance ranking, without any approximate-search error. Of 10,000 synthetic vectors, 500 are authorized. Fetching 10 candidates from the entire collection and filtering leaves 0. Fetching 100 leaves 5.

Simulation table: fetching 10, 50, 100, 200, and 500 candidates from the entire collection leaves 0, 2, 5, 10, and 10 results after filtering. Recall against the eligible top 10 is 0%, 20%, 50%, 100%, and 100%.
Figure 1. Exact distance ranking can still leave too few results after filtering. Synthetic data with a fixed seed; this is not a speed or quality benchmark of an ANN index. The code near the end generates this exact table.

Pre-filtering identifies the shelves you may access before searching them. If the conditions leave only a small subset, computing exact distances within that subset can make more sense than traversing a collection-wide index. pgvector recommends considering indexes on filter columns and exact search when a condition matches few records[1].

Filtering during search is like a librarian continuing to look after encountering an ineligible book, rather than stopping after the first ten. Implementation depends on the system. pgvector offers iterative scans for approximate indexes: keep searching when there are too few results, but stop at resource limits. Enabling this mode therefore does not guarantee ten results every time[1].

One tactic is to fetch extra candidates. If roughly 5% are eligible, you might consider about 200 candidates to obtain ten results. But that calculation is only an expectation when the eligible fraction near the query resembles the estimate. Permissions and topics are often correlated: documents from another company may occupy almost the entire nearest region. Do not turn a heuristic into a guarantee.

Test the index under real conditions: individual companies, permission groups, and document states. Also distinguish two failures. Returning an unauthorized result is a security failure; returning too few results when eligible documents exist is a retrieval failure. Fetching extra candidates can help the latter. It does not replace controls against the former.

HNSW or IVF: what resources buy better recall?

Exact search measures the distance to every vector in the eligible set, like checking every book on the correct shelf. It provides a ground truth for comparison. For large sets, approximate nearest neighbor search (ANN) skips some work to reduce cost, accepting that it may miss some of the nearest neighbors[1].

HNSW organizes vectors into a multilayer graph, like taking a main road to a neighborhood before turning onto smaller streets. Sparse layers enable long jumps; denser layers support a closer search[2]. Construction parameters control connectivity and the search effort spent inserting points. Query-time parameters control how many candidate paths are retained during search. A larger budget generally improves recall, but requires more work and memory.

IVF divides vectors into clusters, like sorting books into topic rooms. A query selects some of the nearest rooms, then checks the books inside them. pgvector’s IVFFlat needs data to learn its clusters. Visiting more clusters at query time improves recall but increases the work. The pgvector documentation describes IVFFlat as having lower build costs and memory use than HNSW, while HNSW offers a better speed-recall trade-off in that implementation[1].

This is not a universal ranking across datasets. Measure the system while it handles updates and queries together. If new data drifts away from the learned clusters, an older IVF configuration may become less suitable. HNSW does not require that clustering step, but insertions, deletions, and graph maintenance still have costs. “Supports inserting points” does not mean “never needs maintenance.”

Under the assumption of 10 million vectors with 768 float32 values each, the coordinates alone need 30.72 billion bytes, or about 30.7 GB in decimal units. That excludes graph links, filtering metadata, content, replicas, and spare capacity during rebuilds. Do not size machines using raw vector storage alone. Compressing coordinates can save space, like replacing a detailed record with a short code. But measure quality again, along with the cost of reranking against original data if that is part of the design.

Why does updating one chunk require version management?

The source document changes from “refund within seven days” to “refund within three days.” The new embedding job takes longer than expected. If the system writes the new text but keeps the old vector, it searches with coordinates for one sentence and returns the content of another.

One design uses the document store as the source of truth, with a derived vector store serving search. Change events pass through a durable queue, like work tickets kept until they are processed. Writing the source and issuing the ticket must avoid losing events. The article on inverted indexes and the search update pipeline covers that mechanism.

Each job carries a document identifier and version. A worker reads the correct source version, splits it into chunks, creates vectors, and writes them with keys determined by company, document, version, and chunk number. Repeating the same job must not create duplicates. A version 6 job finishing after version 7 must not move the served version back to 6.

When a document has multiple chunks, “the first chunk was written” is not the same as “the document update is complete.” One approach is to build the entire new chunk set, then conditionally switch a pointer to the served version. Queries accept only chunks from the current version, like a librarian replacing the complete instruction pack before changing its label to “current.” If the stores cannot switch together, add version checks and accept temporary missing results during the transition rather than pretending there is a cross-store transaction.

Deleting a document also needs a versioned deletion marker so an old job cannot bring it back. Permission revocation should update the authorization path independently, without waiting for new embeddings. Check current permissions at the point where content is released; if verification fails, do not return it. Cached results must also be scoped to the access context and permission version, or invalidated when permissions change.

Changing the model means changing the map. Build the new space, backfill it, catch up with intervening updates, run queries against both spaces, and compare quality. Switch reads only after the new version meets the requirements. Keep a rollback path for an appropriate period. Do not mix old and new vectors in the same ranking just because their dimensions match.

Can semantic search find a contract identifier?

“When will my money be back?” suits semantic search. An identifier such as “HD-2026-0042” needs exact characters. Two identifiers differing by one digit may look very similar while referring to different records.

Hybrid search is like asking two librarians: one looks up exact words and identifiers, while the other looks up topics. The lexical branch uses an inverted index; the vector branch searches content representations. Both branches must enforce the same permissions, versions, and document status before their results are used.

Their scores are not automatically on the same scale. Do not simply add a lexical score to a vector similarity and call the result relevance. Reciprocal rank fusion (RRF) combines ranks, like scoring positions on two nomination lists instead of directly comparing two kinds of points. Qdrant supports combining candidate sets with RRF in hybrid queries[4].

After merging and deduplication, a reranking model can read the question alongside each passage, like asking someone to inspect the selected books closely. This step has its own cost and can only reorder what was retrieved. A document missed by both branches cannot appear through reranking. Identifiers may need a separate exact-lookup path rather than a hope that semantic search will handle every query.

When do you need more machines, and when do you need better measurements?

Initially, PostgreSQL with pgvector can keep content, conditions, and transactions in one system. It is an option to test if it meets the workload and operational requirements, not a promise that it fits every scale. A dedicated system becomes worth considering when measurements reveal index-management, load-distribution, or operational needs that the current architecture struggles to meet.

Sharding splits the collection into parts, like dividing a library among branches. If the company owning the documents is known, queries can be routed to the relevant part. A large company may need several shards. Careless tenant placement can also overload one machine while others sit idle.

If a query must reach several shards, a coordinator collects and merges candidates. With the same metric and exact search, each shard’s top k is sufficient to merge the global top k, provided ties are handled consistently. But ANN on each shard can miss neighbors, and post-filtering on each shard can return too few results. Adding shards does not fix either problem. Overall query latency is also affected by the slowest shard.

A replica is like a copy of a collection kept at a backup branch. Depending on the design, it can help serve load or tolerate failures. In return, replicas consume storage and may not have received the same updates yet. Define when a write is acknowledged, when it becomes searchable, and what a reader might see after switching replicas. Replicas do not replace backups and recovery drills.

Quality needs two separate scorecards. ANN recall@k is the fraction of ground-truth neighbors recovered by approximate search, like comparing ten fetched books with the ten nearest books found by checking the whole shelf. Ground truth must use the same eligible set, version, and metric, not the collection-wide top k before filtering.

The other scorecard uses questions judged for usefulness: does the correct policy passage appear, is it ranked high enough, and does it contain enough context? High ANN recall only shows that the algorithm closely matches the chosen distance calculation. It does not prove that the embedding understands the question well, much less that the eventual answer is correct.

Under load, track time spent embedding the query, retrieving candidates, filtering, reranking, and reading content. Break down measurements by large and small companies, broad and restrictive filters, and identifier queries. Track underfilled results, update lag, version errors, and blocked unauthorized results. One average latency figure can hide every group that is struggling.

Try it yourself: correct ranking, too few results

The Python below uses numpy and matplotlib. It creates 10,000 vectors with 24 dimensions, normalizes their lengths, and ranks the entire collection by dot product. For these normalized vectors, that is also the cosine ranking. Every twentieth record is authorized, independently of the synthetic coordinates.

Ground truth is the ten nearest vectors among the 500 authorized records. The program tries fetching extra candidates from the entire collection before filtering. It deliberately avoids HNSW and IVF to separate the effect of filter ordering from ANN error. Run it from the EyeBlogs project root to generate the figure shown above.

filter_vectors.py
from pathlib import Path
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
rng = np.random.default_rng(2026)
x = rng.normal(size=(10_000, 24))
x /= np.linalg.norm(x, axis=1, keepdims=True)
q = rng.normal(size=24)
q /= np.linalg.norm(q)
allowed = np.zeros(len(x), dtype=bool)
allowed[::20] = True # Exactly 5% of records are authorized.
order = np.argsort(-(x @ q), kind="stable")
k = 10
truth = order[allowed[order]][:k]
rows = []
for budget in [10, 50, 100, 200, 500]:
candidates = order[:budget]
hits = candidates[allowed[candidates]][:k]
recall = len(set(hits) & set(truth)) / len(truth)
rows.append([str(budget), str(len(hits)), f"{recall:.0%}"])
print(f"Fetch {budget:3} candidates: keep {len(hits):2}, recall {recall:.0%}")
# Monochrome table: every value is readable without color encoding.
surface, ink, muted = "#fcfcfb", "#0b0b0b", "#52514e"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 14})
fig, ax = plt.subplots(figsize=(11, 5.7), dpi=160, facecolor=surface)
ax.set_axis_off()
table = ax.table(
cellText=rows,
colLabels=["Candidates fetched", "After filtering (up to 10)", "Eligible top-10 recall"],
cellLoc="center", colLoc="center", bbox=[0.02, 0.14, 0.96, 0.70],
)
table.auto_set_font_size(False)
table.set_fontsize(13)
for (row, col), cell in table.get_celld().items():
cell.set_facecolor(surface)
cell.set_edgecolor("#e1e0d9")
cell.set_linewidth(0.8)
cell.set_text_props(color=ink, weight="bold" if row == 0 else "normal")
fig.suptitle("Exact distances, not enough results", y=0.95,
fontsize=21, color=ink, weight="bold")
fig.text(0.5, 0.84, "10,000 synthetic vectors · 500 authorized · 10 results wanted",
ha="center", fontsize=13, color=muted)
fig.text(0.5, 0.075, "Rank the entire collection exactly, then filter by access.",
ha="center", color=muted, fontsize=13)
fig.text(0.5, 0.025, "Seed 2026 · Not an ANN benchmark or real embedding data",
ha="center", color=muted, fontsize=11)
out = Path("images/vector-database")
out.mkdir(parents=True, exist_ok=True)
fig.savefig(out / "loc-ung-vien-en.png", facecolor=surface)
plt.close(fig)

The runs leave 0, 2, 5, 10, and 10 records, respectively. In this run, fetching 200 candidates recovers the entire eligible top 10. You cannot infer a rule that “fetching 20 times as many is enough.” Changing the seed, authorized fraction, or correlation between permissions and topics changes the results.

This experiment computes distances and sorts the entire dataset before cutting the candidate list. The table therefore only demonstrates that truncating candidates before filtering can lose answers. It does not show that fetching 200 is faster than scanning the 500 eligible records. To choose a real configuration, test a real index with a real workload and real filters.

Back to “When will my money be back?”

A good search box does more than recognize that “money back” is close to “refund.” It also knows which collection the employee is asking about, which policy is current, and which passages they may read. If permissions cannot be verified, no result is better than an excerpt that should never have appeared.

A vector database makes a new kind of lookup possible. But getting the right document to the right person remains a system-wide problem. The model supplies coordinates, the index retrieves candidates, and the update pipeline maintains versions. The final measurement checks whether the reader actually receives an answer.

Free geometry illustrations for students and teachers: EyeViz.

Tài liệu tham khảo

  1. [1]pgvector: Indexing, Filtering, Multitenancy and Iterative Index Scans. pgvector.
  2. [2]Yu. A. Malkov and D. A. Yashunin. Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020.
  3. [3]Vectors: Named vectors. Qdrant Documentation.
  4. [4]Hybrid and Multi-Stage Queries. Qdrant Documentation.