The Problem Isn't Embedding 1 Million Documents. It's Protecting What's Inside Them.

Embedding one million documents is an engineering challenge. Embedding one million documents without unnecessarily exposing sensitive customer information is a security challenge.

Enterprise documents contain far more than business knowledge. They contain customer names, email addresses, phone numbers, financial records, account numbers, healthcare information, and confidential conversations.

When building a Retrieval-Augmented Generation (RAG) pipeline, these documents are extracted, chunked, and sent to embedding models to generate vector representations.

But there is an important question most RAG architectures overlook:

Why should an embedding provider receive a customer's actual personal information just to understand the semantic meaning of a document?

At Hrida AI, we approached this differently.

Instead of attempting to secure sensitive information after embeddings have already been generated, we introduced a privacy enforcement layer before vectorization using SPRO, our AI/LLM Gateway with guardrails enabled by default.

The approach uses PII detection and masking with typed placeholders to ensure that sensitive identifiers are replaced before content reaches the embedding model.

This is how we approach privacy-conscious document processing at scale.

Why Embedding Customer Documents Creates a Privacy Challenge

A conventional RAG ingestion pipeline looks straightforward:

1. Upload documents.
2. Extract text.
3. Split content into chunks.
4. Generate embeddings.
5. Store vectors in a vector database.
6. Retrieve relevant chunks during queries.

The problem appears between steps 3 and 4.

Consider a customer support document:

Jane Doe contacted the support team using
jane.doe@company.com regarding account number
8291045621.

Jane reported an unauthorized transaction.

When this content is sent directly to an external embedding provider, the provider receives the original personal identifiers.

The resulting embedding is not a readable copy of the document. However, embeddings should not automatically be considered anonymous or privacy-safe. Research has demonstrated that information about original text can sometimes be recovered from embeddings.

Encrypting the vector database does not prevent sensitive information from being exposed during the embedding request itself.

This creates a gap in conventional RAG security.

The data needs protection before it reaches the embedding model, not just after it enters the vector database.

The Missing Layer: Privacy Before Vectorization

Most RAG security discussions focus on protecting stored vectors, controlling retrieval permissions, and preventing unauthorized access.

Those controls are important, but they address different stages of the pipeline.

We wanted to introduce another layer:

Inspect and mask sensitive information before sending content to an embedding provider.

This is where SPRO comes in.

SPRO is Hrida AI's unified AI/LLM Gateway, providing model routing and built-in security guardrails across AI requests.

Its sensitive-information guardrails support detection and masking of:

• Personally Identifiable Information (PII).
• Protected Health Information (PHI).
• Personal Financial Information (PFI).
• API keys and other sensitive credentials.

By routing embedding requests through SPRO, applications can apply configured privacy guardrails before content is forwarded to the embedding provider.

This eliminates the need to independently implement and maintain a separate masking integration for every embedding provider.

Traditional RAG vs Privacy-First RAG

The key difference is when sensitive information is handled.

In the privacy-first approach, masking happens before the embedding model receives the content.

Typed Placeholders: Removing Identity Without Losing Context

Simply replacing every detected value with [REDACTED] is not always ideal.

Consider this document:

Jane Doe contacted support using
jane.doe@company.com regarding account
8291045621.

Jane reported an unauthorized transaction.

A generic masking approach might produce:

[REDACTED] contacted support using [REDACTED]
regarding account [REDACTED].

[REDACTED] reported an unauthorized transaction.

The sensitive values are hidden, but the document loses useful structural information.

Is the first redacted value a person, an organization, or a location? Is the second an email address or a phone number?

Typed placeholders preserve the entity category.

With typed masking, the representation becomes:

[PERSON_1] contacted support using
[EMAIL_1] regarding account
[ACCOUNT_NUMBER_1].

[PERSON_1] reported an unauthorized transaction.

Notice the difference.

The actual identifiers are removed, but the semantic structure remains.

Why Typed Placeholders Matter

1. Context preservation

The embedding model can still understand that a person contacted support, used an email address, and reported an account-related issue.

2. Entity differentiation

A person, email address, phone number, and account number remain distinguishable.

3. Consistent references

When the same entity appears multiple times, consistent placeholders such as [PERSON_1] help preserve references within the processed content, provided the masking implementation maintains consistent mappings.

4. Reduced unnecessary exposure

The embedding provider receives contextual information without requiring the original personal identifiers.

This is an important distinction: masking should not destroy the information needed for semantic retrieval.

How We Integrated SPRO Into the RAG Embedding Pipeline

The architectural change is relatively simple.

Instead of sending extracted document chunks directly to an embedding provider, we route embedding requests through SPRO.

The resulting pipeline looks like this:

Let's break down the important stages.

1. Document Extraction and Chunking

Documents are first processed through the ingestion pipeline.

This can include PDFs, Word documents, spreadsheets, and other enterprise file formats.

The extracted content is divided into manageable chunks while retaining document identifiers and relevant metadata.

2. Route Embedding Requests Through SPRO

Instead of calling the embedding provider directly, the application sends embedding requests through SPRO.

SPRO applies the configured security guardrails to inspect the incoming content.

Detected sensitive information is masked before the request proceeds to the embedding provider.

3. Generate Embeddings From Masked Content

Once the content has been processed, the embedding model receives the sanitized representation.

For example:

Original:
Jane Doe from Chennai contacted
jane.doe@company.com about her account.

Masked:
[PERSON_1] from Chennai contacted
[EMAIL_1] about her account.

The embedding model processes the masked representation rather than the original personal identifiers.

4. Store the Generated Vectors

The generated embeddings can then be stored in the vector database alongside appropriate document and tenant metadata.

For PostgreSQL and pgvector-based implementations, this allows vector representations and associated metadata to be managed within the same database infrastructure.

5. Apply Access Controls During Retrieval

Privacy at embedding time does not replace retrieval security.

When a user submits a query, the application must still enforce tenant-level and document-level permissions before returning relevant chunks.

The retrieval layer should ensure that customers can access only the documents they are authorized to see.

What Happens to the Original Customer Information?

An important distinction is that masking is not the same as deleting the original document.

The original document may still need to be retained for business operations, auditing, or authorized document retrieval.

However, the embedding pipeline can operate on a masked representation.

If an application needs to restore original identifiers later, it must maintain a separately protected mapping mechanism with appropriate access controls.

The placeholder itself should never be treated as a secure storage mechanism.

This separation gives us two distinct responsibilities:

The objective is to minimize the sensitive information exposed to downstream AI services.

Scaling Privacy-First Embedding to 1 Million Documents

Now consider the same architecture at enterprise scale.

Processing one million documents introduces challenges around throughput, concurrency, retries, consistency, and storage.

A privacy layer must work without becoming an obstacle to the ingestion pipeline.

We approach this by separating the process into manageable stages.

There are a few important engineering considerations here.

Batch processing: Large document collections should be processed in controlled batches rather than as one massive request.

Failure recovery: Failed embedding requests should be retried without unnecessarily repeating successful processing.

Incremental updates: When documents change, only affected content should need reprocessing.

Deletion propagation: Removing a document should also address its associated chunks, embeddings, indexes, and relevant cached content.

Observability: Monitor processing throughput, guardrail decisions, embedding latency, failures, and provider usage without unnecessarily logging original sensitive content.

The objective is not simply to process more documents. It is to maintain the same privacy enforcement throughout the ingestion lifecycle.

Why Use an AI Gateway Instead of Building Separate PII Middleware?

One option is to build a custom PII detection service and integrate it individually with every embedding provider.

That approach can work, but it creates additional engineering responsibilities.

Every new provider integration, model migration, or embedding endpoint becomes another place where security controls need to be implemented and maintained.

With SPRO, the gateway becomes the centralized enforcement point.

The important advantage is architectural consistency.

Security policies can be enforced at the gateway rather than being duplicated across every application and provider integration.

What This Does Not Solve: Security Beyond PII Masking

PII masking before embedding is an important security control, but it is not a complete enterprise security architecture.

There are several additional considerations.

Tenant isolation: Masking personal information does not prevent one customer from retrieving another customer's documents. Tenant-aware authorization must still be enforced.

Sensitive information detection: No detector should be assumed to identify every possible sensitive value. Custom identifiers and domain-specific information may require additional patterns or policies.

Semantic leakage: Masked documents can still contain confidential business information. Removing names and account numbers does not make the remaining content public.

Prompt injection: Documents can contain malicious instructions. Retrieved content must be treated as untrusted input and protected through appropriate prompt-injection defenses.

Logging and retention: Original documents, request payloads, mappings, and diagnostic logs must follow appropriate retention and access policies.

Deletion and synchronization: Document updates and deletion must propagate to all derived representations.

At Hrida AI, we see these as complementary layers of a broader enterprise AI security architecture.

Key Takeaways

• Privacy starts before embedding: Protecting a vector database is not enough if raw customer information has already reached an embedding provider.
• Typed placeholders preserve context: Replacing identifiers with entity-specific placeholders retains useful semantic structure.
• SPRO provides the enforcement layer: Built-in AI guardrails can inspect and mask sensitive information before embedding requests reach downstream providers.
• Scale should not weaken privacy: The same security enforcement needs to apply across the document ingestion pipeline.
• PII masking is one layer, not the entire security model: Tenant isolation, access controls, encryption, retention, and secure retrieval remain essential.

Frequently Asked Questions

Can embeddings contain personally identifiable information?

Embeddings are numerical representations rather than directly readable copies of their source documents. However, research has demonstrated that information can sometimes be recovered from embeddings. They should therefore be treated as potentially sensitive data.

Why mask PII before generating embeddings?

Masking before embedding reduces unnecessary exposure of original personal identifiers to downstream embedding providers. It also reduces the amount of sensitive information entering the vectorization pipeline.

What are typed placeholders?

Typed placeholders replace detected sensitive values with entity-specific identifiers such as [PERSON_1], [EMAIL_1], and [PHONE_1]. This preserves useful structural context without retaining the original identifiers in the masked text.

Does PII masking affect RAG retrieval quality?

It can affect retrieval quality depending on the document and the information being removed. Typed placeholders help preserve entity categories and surrounding context, but retrieval quality should be evaluated against representative datasets.

Can SPRO be used for embedding requests?

SPRO supports embedding API endpoints, allowing applications to integrate embedding workflows through its gateway and apply configured security controls.

Does PII masking eliminate the need for tenant isolation?

No. PII masking and tenant isolation solve different problems. Masking reduces exposure of sensitive identifiers, while tenant isolation ensures that users can access only the information they are authorized to retrieve.

Can this approach work with multiple embedding providers?

SPRO provides a unified gateway for supported embedding and AI providers. This allows applications to integrate through a common gateway rather than maintaining entirely separate security integrations for each provider.

Conclusion: Privacy Before Vectorization

The next generation of enterprise AI will not be measured only by the number of documents it can process or the speed at which it retrieves information.

It will also be measured by how responsibly it handles the information entrusted to it.

Processing one million documents is impressive.

But ensuring that sensitive customer identifiers are masked before they reach an embedding provider is an equally important engineering decision.

With SPRO, Hrida AI brings security guardrails directly into the AI request lifecycle, making privacy enforcement part of the embedding architecture rather than an afterthought.

Don't wait until retrieval to protect customer data. Protect it before it becomes an embedding.

🚀 Explore SPRO

Looking to introduce privacy controls into your enterprise AI and RAG pipelines?

SPRO provides a unified AI gateway with built-in guardrails, intelligent model routing, and embedding API support.

🌐 Learn More

• 🛡️ SPRO Product: https://spro.hridaai.com/
• 🔐 SPRO Security: https://spro.hridaai.com/
• 🏢 Hrida AI: https://www.hridaai.com/
• 📚 Technical Blogs: https://www.hridaai.com/blogs/spro
• 📅 Book a Demo: https://www.hridaai.com/book-demo

Your data should power your AI. It shouldn't become its next privacy risk.