Day 2: RAG Architectures

Module Overview

Session: RAG Architectures

• Simple RAG & Memory
• Advanced Retrieval Strategies
• Agentic & Self-Correcting RAG

Today we’re exploring the landscape of Retrieval Augmented Generation architectures. We’ll move beyond the basic “retrieve and generate” loop to see how modern systems handle complexity.

We’ll cover 8 distinct architectures, ranging from simple setups to complex, agentic systems that can self-correct and reason.

Session: RAG Architectures

What You’ll Learn Today

By the end of this session, you will:

• Identify 8 key RAG architectures and their use cases
• Understand when to add memory, branching, or agents to RAG
• Recognize how to optimize retrieval for complex queries

By the end of this hour, you’ll be able to look at a problem and decide which RAG architecture fits best.

You’ll understand not just how they work, but why you would choose one over the other. Whether you need speed, accuracy, or the ability to handle multi-step reasoning, you’ll know the right pattern to use.

1. Simple RAG

Simple RAG workflow

• Workflow: Query → Retrieve → Generate
• Use Case: FAQ systems, limited knowledge bases
• Pros: Simple, fast, deterministic retrieval

This is the baseline. 1. Simple RAG.

The model retrieves documents from a static database based on a user query and generates an answer. It’s straightforward and works well when your database is small and your queries are direct.

Think of it like a librarian looking up a book and summarizing it for you. It’s great for FAQs where the answer is definitely in the text.

2. Simple RAG with Memory

Simple RAG with Memory workflow

• Addition: Conversation history
• Workflow: Query + History → Retrieve → Generate
• Use Case: Customer service chatbots, personalized recommendations

Next is 2. Simple RAG with Memory.

This adds a storage component for conversation history. The model uses past interactions to provide context.

If a user says “I have a problem with my order,” and then follows up with “It hasn’t arrived,” the model needs memory to know “It” refers to the order. This is essential for any conversational bot.

3. Branched RAG

Branched RAG workflow

• Concept: Route query to specific sources
• Workflow: Query → Classifier → Specific DB → Generate
• Use Case: Legal tools, multidisciplinary research

3. Branched RAG. This is about routing.

Instead of searching everything, the system first decides where to look. If the user asks a legal question, it routes to the legal database. If they ask a technical question, it routes to the documentation.

This reduces noise. You don’t want medical advice showing up in your legal search results. It selects the most relevant source based on the input.

4. HyDe (Hypothetical Document Embedding)

HyDe workflow

• Concept: Generate answer first, then retrieve
• Workflow: Query → Hypothetical Answer → Embed & Retrieve → Generate
• Use Case: Vague queries, creative content, R&D

4. HyDe - Hypothetical Document Embedding.

This is a clever twist. Instead of searching for the user’s question, the model first hallucinates a hypothetical answer. It then uses that hypothetical answer to search for real documents.

Why? Because the user’s question might not look like the answer document. But a hypothetical answer will look like the real answer. It bridges the semantic gap between query and document.

5. Adaptive RAG

Adaptive RAG workflow

• Concept: Adjust strategy based on complexity
• Workflow: Query → Complexity Analysis → Strategy A/B/C → Generate
• Use Case: Enterprise search, varying query difficulty

5. Adaptive RAG.

This is about efficiency. Not every query needs a cannon.

Adaptive RAG analyzes the complexity of the query first. - Simple question? Do a quick keyword search. - Complex analysis? Do a multi-step semantic search.

It balances speed and depth by adapting its strategy in real-time.

6. Corrective RAG (CRAG)

Corrective RAG workflow

• Concept: Evaluate retrieval quality
• Workflow: Query → Retrieve → Grade Documents → (Web Search if poor) → Generate
• Use Case: High-accuracy domains (Medical, Legal)

6. Corrective RAG (CRAG).

This adds a “grader” step. After retrieving documents, the system checks: “Are these actually relevant?”

If they are good, proceed. If they are bad, CRAG might trigger a fallback, like a web search, to find better information. It filters out irrelevant “knowledge strips” to ensure the generation is based on high-quality data.

7. Self-RAG

Self-RAG workflow

• Concept: Model self-queries during generation
• Workflow: Generate → Critique → Retrieve More → Generate
• Use Case: Long-form content, exploratory research

7. Self-RAG.

Here, the model is more active. As it generates an answer, it might realize: “Wait, I need more info on this specific point.”

It can pause, issue a new retrieval query to fill that gap, and then continue generating. It’s an iterative process of generation and retrieval, allowing for much more comprehensive answers.

8. Agentic RAG

Agentic RAG workflow

• Concept: Agents orchestrate retrieval
• Workflow: Meta-Agent → Document Agents → Synthesis
• Use Case: Complex research, multi-source aggregation

Finally, 8. Agentic RAG.

This is the most complex. You have a “Meta-Agent” that manages other agents. You might have “Document Agents” that are experts on specific documents.

The Meta-Agent breaks down a complex user request, assigns tasks to sub-agents, and synthesizes their findings. It’s like a project manager coordinating a research team. It can handle multi-step, complex reasoning tasks.

Summary

Key Takeaways

• Start Simple: Standard RAG works for many use cases
• Add Complexity only when needed: Memory, routing, or correction
• Agentic patterns allow for reasoning and multi-step tasks
• HyDe helps when queries are vague

To wrap up:

1. We started with Simple RAG - the foundation.
2. We added Memory for conversation.
3. We used Branching for efficiency.
4. We saw HyDe specifically for vague semantic matching.
5. Adaptive RAG gave us efficiency routing.
6. Corrective RAG gave us quality control.
7. And Self-RAG and Agentic RAG gave us autonomy and reasoning.

Choose the architecture that matches your specific problem constraint.