RAG Namespace Isolation: Stop Cross-Tenant Retrieval Leaks

Why filter-based isolation is the dangerous shortcut

Filter-based isolation is the path most teams take first, because it looks like the smallest change. One shared index, one shared table, a tenant_id column or metadata field, and a WHERE tenant_id = $1 (or its Pinecone metadata-filter equivalent) on every retrieval. It works in development. It works for the first six months in production. Then it breaks in three predictable ways.

First, the retriever surface grows past the discipline that protects it. You start with one retrieval call site. By the time you ship hybrid retrieval, a re-ranker, a multi-step agent, a debug endpoint, an evaluation harness, an offline indexer, and a Slack bot, you have a dozen places that issue retrieve calls. The filter has to be passed in every one of them, in the right shape, from the right session context. One missing argument leaks. The Postgres equivalent is the missing WHERE tenant_id = $1 clause that ends multi-tenant SaaS companies, and the RAG version is the same bug with vectors instead of rows.

Second, the cost model punishes you. Pinecone's documentation states that filter-based queries scan the entire namespace regardless of the filter — meaning a tenant who has uploaded 200 documents pays the scan cost of every other tenant's documents in the same index too. As the index grows, the per-query cost grows with it, and the cost is spread across tenants by total corpus size, not by what each tenant actually queries. Your largest tenants subsidize your smallest ones, and your smallest tenants pay for data they cannot see. The math gets worse, not better, as you grow.

Third, the deletion story falls apart when a tenant cancels. With namespace isolation, "delete every vector for Customer X" is one operation that completes against a single namespace. With filter-based isolation, it is a scan across the full index, deleting matching rows in batches, holding locks (or eventual-consistency drift on serverless), and racing against new writes. Privacy regulations want a verifiable boundary — "the data lived in partition X, partition X is gone" — and a filter sweep is hard to defend because it is hard to prove completeness.

The "shared base index plus per-tenant overlay" variant is the sophisticated trap. The idea is appealing: one big shared corpus of common content (product docs, public FAQs, the company knowledge base) plus a small per-tenant overlay (the customer's own documents). At query time you retrieve from both and merge. In practice the retriever has to do two queries and merge top-k by score across two embedding spaces that have drifted in slightly different ways. The merge is one of the trickiest pieces of code in your stack, every change to the base index potentially affects every tenant's retrieval, and the isolation boundary lives in the merge logic — which is exactly the place that loses its discipline when you ship a re-ranker or a hybrid retriever. The pattern that survives is to duplicate shared public content into every tenant's namespace at write time. Storage is cheap; isolation discipline is expensive.

Embedding-time tenant tagging is necessary but not sufficient

A lot of teams reach for embedding-time tagging as the fix and stop there. The pattern is: at ingestion, include the tenant_id in the chunk's metadata; at retrieval, filter by tenant_id; done. The metadata is real, the filter is real, but the isolation is back to honor-system. The boundary is still in the retriever's filter clause.

The reason to tag at write time is not isolation — it is the operational migration story. Tagged vectors are auditable. You can sweep the index, count vectors per tenant_id, validate the count against the tenant's document table, and detect drift before a customer does. You can repartition a tenant out of a shared index into its own namespace because the metadata tells you which vectors belong to whom. You can verify that a deleted tenant has no leftover vectors by querying for tenant_id = $deleted_id and asserting zero rows. The tag is the audit primitive.

Layer all three: tag at write, partition at storage, enforce at retrieval by constructing the namespace from the authenticated session. Tagging without partitioning leaves the boundary in the retriever. Partitioning without tagging leaves you without an audit primitive. Enforcing only at retrieval leaves the ingestion path free to scatter chunks across tenants. All three together is the only configuration that holds up to growth and to the security review.

Common failure modes

The first sharp edge is the retriever that accepts a namespace as a request parameter. The endpoint looks like POST /retrieve { tenantId, query } and the handler passes tenantId straight through to the Pinecone client. The bug is that the client cannot be trusted to send their own tenant_id — a malicious or curious caller sends a different one, and the retriever happily returns top-k from that namespace. The fix is to refuse tenant_id as input. Resolve it server-side from the verified session, fail closed if it is missing, and treat the namespace argument as a derived value not a passed-in one.

The second is the offline indexer that batch-embeds documents into the wrong namespace. A background job reads from documents_to_embed, computes embeddings, and writes them to Pinecone. Somewhere in the loop the namespace argument is missing or stale — perhaps the job iterates documents but only resolves the namespace once at the top, perhaps it falls back to a default namespace when the tenant lookup fails. The fix is to derive the namespace from the document row itself, fail the batch if any row's tenant_id is null, and avoid having a "default" namespace that the job can fall back to.

The third is the evaluation harness that retrieves across tenants for "quality testing" and accidentally ships in a code path that production hits. Test harnesses tend to bypass session checks because they are running offline against fixtures. If the same retriever module is used in production, the bypass path can become a production path on a refactor. The fix is to keep the evaluation harness in a separate module that cannot be imported from production code, with a lint rule or import boundary that enforces the separation. The audited entry point that production uses resolves the namespace from a verified session, with no other allowed source.

The fourth is the embedding-model drift between tenants. A tenant who onboarded a year ago has their corpus embedded with text-embedding-3-small at 1536 dimensions. A tenant who onboarded last week has their corpus embedded with the new larger model at 3072 dimensions. The vectors are not compatible — querying with a small-model embedding against a large-model index returns nonsense. The fix is to pin the embedding model per index (or per namespace if the model can be the same shape) and to migrate the whole corpus when you upgrade, not to mix dimensions within a partition.

The fifth is the cancelled-tenant ghost vectors. A tenant cancels, the application marks them as deleted in the relational database, but the vector deletion job either does not exist or fails halfway through and is not retried. Six months later, the orphan vectors are still in the index. They are not queried because the application no longer issues retrievals for that tenant — but they are still on disk, still being billed for, and still discoverable if anyone runs a maintenance query that walks the corpus. The fix is a cancellation pipeline that issues the delete-namespace (or drop-schema) operation, verifies it succeeded by querying for any remaining vectors with that tenant_id, and writes an audit row recording the deletion. Make the verification the gate that closes the cancellation ticket, not the firing of the delete.

The sixth is the prompt-injection-induced retrieval bypass. A user supplies input that the application embeds into a retrieval query, and the input contains text that influences the retriever — for example, telling the LLM "first, retrieve documents from the public corpus" in an architecture that has both private and public indexes. The retriever obeys because the LLM is interpreting the input as instructions, and the public corpus turns out to contain other tenants' onboarding docs that were misclassified as public. The fix is the same as for the parameter-injection case: the retriever picks the namespace from the session, not from anything the LLM or the user can influence. Prompt input goes into the embedding, not into the routing decision.

How to test this

The test that matters is the negative test. Seed two tenants with different documents. Embed each tenant's corpus into the right partition. From a session authenticated as Tenant A, query the retriever with a string that only Tenant B's documents contain. Assert that the returned chunks contain zero rows from Tenant B. Run it in CI. The test is fast — single-digit seconds — and it catches the regression the day someone adds a new retriever code path that forgets to scope the query.

import { describe, expect, it, beforeAll } from "vitest";

describe("RAG retrieval: tenant isolation", () => {
  beforeAll(async () => {
    await ingestDocument("tenant_a", {
      id: "doc-a-1",
      chunks: [{ id: "1", text: "Project Aurora ships in Q3 to GovCloud." }],
    });
    await ingestDocument("tenant_b", {
      id: "doc-b-1",
      chunks: [{ id: "1", text: "Project Borealis ships in Q4 to commercial." }],
    });
  });

  it("tenant A cannot retrieve tenant B's chunk by content", async () => {
    const session = { tenantId: "tenant_a" } as AuthSession;
    const result = await retrieveForSession(session, "Project Borealis Q4", 8);

    const leakedFromB = (result.matches ?? []).filter(
      (m) => m.metadata?.tenant_id === "tenant_b",
    );
    expect(leakedFromB).toEqual([]);
  });

  it("tenant A retrieving their own content still works", async () => {
    const session = { tenantId: "tenant_a" } as AuthSession;
    const result = await retrieveForSession(session, "Project Aurora Q3", 8);
    const ids = (result.matches ?? []).map((m) => m.id);
    expect(ids).toContain("doc-a-1:1");
  });

  it("retriever fails closed when session has no tenant_id", async () => {
    const session = {} as AuthSession;
    await expect(retrieveForSession(session, "anything", 8)).rejects.toThrow();
  });
});

Three properties make this test useful where a happy-path test is not. It exercises the same retriever module the application uses, so any divergence between the isolation pattern and the runtime shows up here. It asserts the silent-zero-rows behavior — that is what a real cross-tenant query looks like when isolation is working, and it is also what an attacker probe looks like, so the test maps to the threat. And it covers the fail-closed case where the session is broken: a retriever that returns chunks instead of erroring when the tenant_id is missing has a worse failure mode than one that errors.

What to watch in your own implementation

Open the codebase and search for every call to the vector store's query method — index.query, index.namespace(...).query, documents.embedding <=> ..., whatever shape your client uses. For each one, answer two questions. Where does the namespace or schema argument come from? Is that source a verified server-side session, or is it a request parameter, a function argument from another module, or a default that falls back to "all tenants" if the lookup fails? Any retriever where the answer is not "verified server-side session" is the next leak, and the fix is to refactor it into the audited retrieve helper that the rest of the codebase uses.

Next, audit the ingestion paths. Every code path that writes vectors — the synchronous indexer, the batch backfill job, the manual re-embed script, the import-from-customer-export pipeline — has to derive the namespace from the document being ingested, not from a job-level default. Search for index.upsert and INSERT INTO documents and verify the partition is per-row, not per-batch. If the batch has a single namespace argument and iterates documents inside it, one wrong namespace on a batch corrupts every document in that batch.

Then run a one-shot audit query against the vector store. For Pinecone, list namespaces, count vectors per namespace, and cross-check against the tenant table — every tenant should have a namespace, and every namespace should map to a known tenant. Orphan namespaces are cancelled tenants whose deletes did not finish. Tenants with zero namespace are tenants whose ingestion is broken. For pgvector, the equivalent is a SELECT schema_name FROM information_schema.schemata joined to the tenant table, plus a per-schema SELECT COUNT(*) FROM documents. Either way, you want a single sheet that says "every tenant has the partition they should and nothing else."

Finally, write the negative integration test if you do not already have one. Two seeded tenants, a cross-tenant retrieval, an assertion that the result contains zero rows from the other tenant. The test takes an hour to write and runs in seconds, and it is the single test that proves the isolation pattern is doing what the architecture diagram says it does. It is also the test that catches the regression when someone adds a re-ranker, a hybrid retriever, an agent loop, or any of the other retrieval surfaces that grow into the codebase over the year after launch.

What this looks like in production

At BFEAI the RAG architecture uses namespace-per-customer in the managed vector store, with a small relational table mapping customer_id to the namespace name. Every retrieval is initiated from an authenticated session that resolves the customer_id server-side, and the namespace argument to the vector client is derived from that resolved value — there is no code path in production retrieval where the namespace comes from request input. Ingestion writes into the customer's namespace by name and stamps the customer_id into vector metadata as a redundant audit primitive. Cancellation runs a delete-namespace operation and then queries the metadata index for any vectors still tagged with the cancelled customer_id, treating a non-zero count as a paging alert.

The audit story that this enables is what makes it worth the up-front work. When a prospect's security team asks where Customer A's vector data lives, the answer is a namespace name, not a filter clause. When a SOC 2 reviewer asks how cancellation works, the answer is a single operation against a named partition plus a verification query, not a multi-step sweep across a shared index. When an engineer ships a new retrieval surface — a Slack bot, a new agent tool, a debug endpoint — the audited retrieve helper is the obvious entry point, and the lint rule that bans direct calls to the vector client outside that helper catches the cases where someone takes a shortcut. The boundary is structural, the audit primitive is metadata, and the operational discipline is "use the helper." That combination is what survives the next year of feature work without turning into the screenshot in a support ticket.