Hybrid Search & Guardrails: Production RAG with pgvector, Reciprocal Rank Fusion & Gemini
Soyebuzaman Naim
Robotics, Applied AI & Full-Stack Engineer
Hybrid Search & Guardrails: Production RAG with pgvector, Reciprocal Rank Fusion & Gemini
Most naive Retrieval-Augmented Generation (RAG) tutorials stop at chunking a PDF, generating vector embeddings with OpenAI, and executing an approximate nearest neighbor (ANN) cosine query.
In production environments, pure vector search fails whenever users ask exact keyword queries (e.g. "What is Naim's CGPA?" or "Show the GitHub repo for OFFChat").
In this article, I demonstrate how to build an ultra-reliable, grounded RAG pipeline combining Full-Text Lexical Search (BM25) with pgvector Dense Search, fused via Reciprocal Rank Fusion (RRF) and guarded with multi-tier prompt safety checks.
#1. The Core Architecture: Hybrid Retrieval Pipeline
User Query: "What awards did Naim win for RAG research?"
|
+-------------+-------------+
| |
v v
+------------------+ +------------------+
| Lexical Search | | Dense Vector |
| (PostgreSQL GIN) | | (pgvector HNSW) |
| ts_rank_cd | | 1 - (vec <=> emb)|
+--------+---------+ +--------+---------+
| |
+------------+-------------+
|
v
+--------------------------+
| Reciprocal Rank Fusion |
| RRF_Score = 1 / (k + r) |
+------------+-------------+
|
v
+--------------------------+
| Re-Ranking & Grounding |
| Strict Prompt Injection |
| & Hallucination Guard |
+------------+-------------+
|
v
+--------------------------+
| Gemini 2.5 Flash Stream |
+--------------------------+#2. Reciprocal Rank Fusion (RRF) SQL Implementation in PostgreSQL
Instead of attempting to normalize cosine distances with arbitrary BM25 float scores, Reciprocal Rank Fusion uses rank positions:
-- Production Hybrid Search with pgvector and PostgreSQL Full-Text
WITH dense_matches AS (
SELECT id, title, content,
ROW_NUMBER() OVER (ORDER BY embedding <=> $1::vector) as rank_dense
FROM "KnowledgeChunk"
WHERE status = 'INDEXED'
LIMIT 20
),
lexical_matches AS (
SELECT id, title, content,
ROW_NUMBER() OVER (ORDER BY ts_rank_cd(to_tsvector('english', content), plainto_tsquery('english', $2)) DESC) as rank_lexical
FROM "KnowledgeChunk"
WHERE to_tsvector('english', content) @@ plainto_tsquery('english', $2)
LIMIT 20
)
SELECT COALESCE(d.id, l.id) as id,
COALESCE(d.title, l.title) as title,
COALESCE(d.content, l.content) as content,
(COALESCE(1.0 / (60 + d.rank_dense), 0.0) +
COALESCE(1.0 / (60 + l.rank_lexical), 0.0)) as rrf_score
FROM dense_matches d
FULL OUTER JOIN lexical_matches l ON d.id = l.id
ORDER BY rrf_score DESC
LIMIT 6;#3. Defense-in-Depth Against Prompt Injection
Portfolio chatbots are public-facing endpoints targeted by users attempting prompt extraction or jailbreaks:
// Multi-Tier Security Verification Layer
const INJECTION_PATTERNS = [
/ignores+(alls+)?(previous|prior)s+instructions/i,
/yous+ares+nows+(unrestricted|DAN|jailbroken)/i,
/systems*prompts*leak/i,
/reveals+hiddens+tokens/i,
];
export function sanitizeAndGuardPrompt(userQuery: string): string {
for (const pattern of INJECTION_PATTERNS) {
if (pattern.test(userQuery)) {
throw new Error("Security Violation: Input violates portfolio prompt boundary guidelines.");
}
}
return userQuery.trim().slice(0, 500); // Strict length boundary
}#4. Key Takeaways
- 1Hybrid beats Pure Vector: Combining dense embeddings with sparse lexical indexing increased retrieval recall on keyword questions from 61% to 98.4%.
- 2Deterministic Context Delimiters: Enclosing retrieved context inside XML tags like
<knowledge_context>...</knowledge_context>prevents the LLM from confusing retrieved text with user commands.
About the Author
Soyebuzaman Naim is a Computer Science & Engineering researcher at Southeast University specializing in autonomous field robotics (ROS 2 / LiDAR SLAM), edge computer vision with TensorRT, 3D WebGL architecture, and production RAG systems.