The RAG patterns I keep reaching for

RAG discourse online is noisy, and every week someone on Twitter announces the new “RAG killer.” In enterprise projects, teams are still debugging chunking strategies and wondering why the model keeps hallucinating about their own documentation.

I’ve been building retrieval pipelines for an enterprise GenAI platform at a global industrial company, covering multimodal document understanding, pgvector at scale and tool-use integration. The mistake I see most often is teams reaching for exotic architectures before they’ve exhausted the basics.

This is the decision tree I use.

Step 1: Start with vanilla vector RAG

Ninety percent of enterprise use cases are solved by:

• A decent embedding model (text-embedding-3-small or -large from OpenAI)
• Reasonable chunking (512–1024 tokens, 10–20% overlap, respect semantic boundaries)
• PostgreSQL with pgvector
• A similarity search with top-k ≈ 5–10
• A prompt that says “Answer using only the provided context; if you can’t, say so.”

If this doesn’t work for your corpus, it is almost always because your chunks are bad.

Step 2: Improve the chunking first

The biggest quality leaps I’ve seen came from:

• Structural chunking: respect headings, paragraphs and list items. Don’t blindly split every 512 tokens.
• Chunk + summary pair: store a small LLM-generated summary of each chunk alongside the chunk itself. Search runs against the summary embeddings and returns the original chunk.
• Metadata filtering: attach document type, source, date and author, and filter on them before the similarity search.
• Query rewriting: use a small model to expand the user query before retrieving.

This is performance you get for free, so do it before you try anything exotic.

Step 3: Hybrid search (sparse + dense)

When your users ask questions with specific IDs, jargon, product codes or acronyms, dense embeddings alone will struggle. Keyword search (BM25, full-text) complements them.

With pgvector, you combine a full-text search score with a cosine similarity score and tune the weight between them.

Step 4: Re-ranking

Retrieve 20 results and re-rank them down to the top 5. A cross-encoder (like bge-reranker-base or a Cohere Rerank API call) decides which results belong in the top-k. It is cheap and worth adding.

Step 5: Consider graph-RAG

Graph-RAG makes sense only if your domain is heavily relational, for example:

• Legal: entity relationships, clause references, citations
• Pharma / healthcare: drug interactions, genes, pathways
• Enterprise knowledge: org charts, approvals, dependencies
• Product catalogs: parts, compatibilities, variants

In domains like these, graph-based retrieval (Neo4j, Neptune, or pgvector + Apache AGE) is worth the extra complexity. Build a knowledge graph and use it to guide the vector search or expand the retrieval context via traversal.

I don’t think most teams will benefit from graph-RAG. If you’re considering it, first prove that vanilla vector RAG is your bottleneck.

Where I’ve landed on our platform

For our platform:

• pgvector on PostgreSQL (we’re Azure-native, already running Flexible Server)
• Per-document-type chunking strategies (PDFs, slides, spreadsheets each get their own)
• Azure OpenAI embeddings (text-embedding-3-large)
• Re-ranking layer for high-stakes queries
• No graph yet, because vanilla + good chunking + re-ranking cleared the bar

This setup is the right architecture for our problem. If we hit a use case where graph traversal helps, we’ll add it then.

Summary

Start with simple vector RAG and good chunks, then add hybrid search and re-ranking. Only reach for a graph if your domain needs it, because your ops team will eventually have to operate every layer of complexity you add.

---

If you’re building enterprise RAG and aren’t sure whether you’ve over- or under-engineered it, I do architecture reviews. Book 30 minutes and I’ll tell you honestly what I think.