architecture

Pattern: Knowledge Architecture (RAG vs Semantic vs Fine-Tuning)

Decision framework for how agents access knowledge: RAG (retrieval), semantic (full-document context), or fine-tuning (model weights). Includes selection criteria and implementation guidance.

Overview

The fundamental question: How does the agent access knowledge?

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                        KNOWLEDGE ACCESS SPECTRUM                        β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚                                                                         β”‚
β”‚  PROMPTING          RAG              SEMANTIC           FINE-TUNING    β”‚
β”‚  (In-context)       (Retrieval)      (Full-doc)         (Weights)      β”‚
β”‚                                                                         β”‚
β”‚  ◄─────────────────────────────────────────────────────────────────►   β”‚
β”‚                                                                         β”‚
β”‚  Fastest setup      Best balance     Large context      Most effort    β”‚
β”‚  Most flexible      Most common      Most coherent      Most rigid     β”‚
β”‚  No persistence     Good for Q&A     Expensive          Best latency   β”‚
β”‚  Token-limited      Chunk artifacts  Token limits       Training cost  β”‚
β”‚                                                                         β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Decision Framework

                        β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                        β”‚ How much knowledge      β”‚
                        β”‚ does the agent need?    β”‚
                        β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                                    β”‚
                    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                    β–Ό               β–Ό               β–Ό
            β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”   β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”   β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
            β”‚ Small     β”‚   β”‚ Medium    β”‚   β”‚ Large     β”‚
            β”‚ (<10 docs)β”‚   β”‚ (10-1000) β”‚   β”‚ (1000+)   β”‚
            β””β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”˜   β””β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”˜   β””β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”˜
                  β”‚               β”‚               β”‚
                  β–Ό               β”‚               β–Ό
          β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”      β”‚       β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
          β”‚ SEMANTIC      β”‚      β”‚       β”‚ RAG           β”‚
          β”‚ Full-doc in   β”‚      β”‚       β”‚ Required      β”‚
          β”‚ context       β”‚      β”‚       β”‚               β”‚
          β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜      β”‚       β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                                 β”‚
                    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                    β–Ό                         β–Ό
            β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”         β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
            β”‚ Query-based?  β”‚         β”‚ Synthesis?    β”‚
            β”‚ ───────────── β”‚         β”‚ ───────────── β”‚
            β”‚ RAG           β”‚         β”‚ Hybrid        β”‚
            β”‚ (retrieval)   β”‚         β”‚ RAG + Summary β”‚
            β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜         β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Approach Comparison

Factor Prompting RAG Semantic Fine-Tuning
Setup time Minutes Days Hours Weeks
Knowledge size <10K tokens Unlimited <200K tokens In weights
Update speed Instant Minutes Instant Hours
Cost per query Medium Medium-High High Low
Accuracy Variable High High Highest
Coherence High Medium Highest High
Citations Manual Automatic Manual None

1. Prompting (In-Context Learning)

When to Use

  • Rapidly changing requirements
  • Small, stable knowledge sets
  • Flexibility > consistency
  • Prototyping and testing

Catalog Applications

  • Email tone/style guidance
  • Output format specifications
  • Role-specific instructions
  • Task-specific templates

Example

STYLE_PROMPT = """
You write sales emails with these characteristics:
- Opening: Specific observation about prospect
- Length: Under 150 words
- CTA: Single, low-friction ask
- Tone: Professional but human
- Avoid: "I hope this finds you well", exclamation points
"""

Limitations

  • Consumes context window
  • No dynamic updates without code changes
  • Can't handle large knowledge bases

2. RAG (Retrieval-Augmented Generation)

See ENT-RAG for core concepts.

When to Use

  • Knowledge base > 50K tokens
  • Frequent knowledge updates
  • Need source citations
  • Q&A over documents
  • Domain-specific accuracy critical

Architecture

Query β†’ Embed β†’ Vector Search β†’ Top-K Chunks β†’ LLM β†’ Response

Best Practices

Practice Why Implementation
Chunk wisely Bad chunks = bad retrieval 500-1000 tokens, 10-20% overlap
Embed metadata Context improves ranking Include title, section, date
Hybrid search Keywords + semantics BM25 + vector, reciprocal rank fusion
Rerank results Improve precision Cross-encoder (Cohere, BGE)
Cite sources Trust + audit trail Return chunk IDs with response
Handle "not found" Avoid hallucination Explicit fallback path

Common Failures

Failure Cause Fix
Wrong chunks Poor embedding/chunking Hybrid search, better chunks
Hallucination LLM fills gaps Stricter prompts, citations
Missed info Top-K too low Increase K, reranking
Incoherent Chunks lack context Better boundaries

Catalog Applications

Agent RAG Application
Policy Q&A Answer from policy docs
Incident Analysis Similar past incidents
Competitive Intel Search competitor history
Account Health Similar account patterns

3. Semantic (Full-Document Context)

When to Use

  • Document < 50K tokens
  • Task requires document structure understanding
  • Cross-references within document matter
  • "Read the whole thing" is correct approach

Advantages Over RAG

Factor RAG Semantic
Coherence Chunks may lack context Full context
Structure Lost in chunking Preserved
Cross-refs May miss Captured
Setup Complex Simple

Token Management

def smart_context_builder(data, max_tokens=100000):
    """Prioritize context when approaching limits."""
    context_parts = []
    token_count = 0

    # Priority 1: Recent/critical (always include)
    critical = get_critical_data(data)
    context_parts.append(critical)
    token_count += count_tokens(critical)

    # Priority 2: Supporting data (include if room)
    if token_count < max_tokens * 0.7:
        supporting = get_supporting_data(data)
        context_parts.append(supporting)

    # Priority 3: Historical (summarize if needed)
    remaining = max_tokens - token_count
    history = get_history(data)
    if count_tokens(history) < remaining:
        context_parts.append(history)
    else:
        context_parts.append(summarize(history, remaining))

    return combine(context_parts)

Catalog Applications

Agent Semantic Application
QBR Prep Full account history
Proposal Generator Full RFP document
Meeting Prep Complete CRM record
Contract Review Full agreement text

4. Fine-Tuning

See ENT-FINE-TUNING for core concepts.

When to Use

  • Consistent output format required
  • Latency critical (<500ms)
  • High volume (cost matters)
  • 500+ examples available
  • Task is narrow and stable

Not Suitable When

  • Requirements change frequently
  • Need current/dynamic knowledge
  • <100 examples
  • Need reasoning transparency

Catalog Applications

Agent Fine-Tuning Use
CRM Extractor Email β†’ structured fields
Expense Categorizer Transaction β†’ category
Resume Screener Resume β†’ assessment

Hybrid Architectures

RAG + Semantic Summarization

Query β†’ RAG Retrieval β†’ Chunks β†’ Summarizer β†’ Coherent Context β†’ Generator

Use when: Need both retrieval precision and narrative coherence

Fine-Tuned Extractor + RAG Validator

Document β†’ Fine-Tuned Extractor β†’ Fields β†’ RAG Policy Check β†’ Validated Output

Use when: Need consistent extraction with policy compliance

Tiered Knowledge

                    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
Hot (prompting) ───►│ Style, format   β”‚ <100 tokens
                    β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
Warm (semantic) ───►│ Current context β”‚ <50K tokens
                    β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
Cold (RAG) ────────►│ Full knowledge  β”‚ Unlimited
                    β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Selection Checklist

## Knowledge Architecture Decision

### Requirements
- [ ] Knowledge size: _____ (tokens/documents)
- [ ] Update frequency: _____ (realtime/daily/weekly/rarely)
- [ ] Query type: Search / Synthesis / Extraction
- [ ] Accuracy requirement: Critical / Important / Nice-to-have
- [ ] Citations needed: Yes / No
- [ ] Budget per query: $_____

### Recommendation
Based on above:
- [ ] Prompting (small, static, flexible)
- [ ] RAG (large, searchable, citable)
- [ ] Semantic (medium, coherent, structured)
- [ ] Fine-tuning (consistent, fast, high-volume)
- [ ] Hybrid: _____

### Implementation Path
See: [Implementation Guide: RAG Systems](/docs/MPD-TEC-IMPL-RAG-01.html) or relevant guide