architecture
Pattern: Knowledge Architecture (RAG vs Semantic vs Fine-Tuning)
Decision framework for how agents access knowledge: RAG (retrieval), semantic (full-document context), or fine-tuning (model weights). Includes selection criteria and implementation guidance.
Overview
The fundamental question: How does the agent access knowledge?
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β KNOWLEDGE ACCESS SPECTRUM β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β β
β PROMPTING RAG SEMANTIC FINE-TUNING β
β (In-context) (Retrieval) (Full-doc) (Weights) β
β β
β βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββΊ β
β β
β Fastest setup Best balance Large context Most effort β
β Most flexible Most common Most coherent Most rigid β
β No persistence Good for Q&A Expensive Best latency β
β Token-limited Chunk artifacts Token limits Training cost β
β β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
Decision Framework
βββββββββββββββββββββββββββ
β How much knowledge β
β does the agent need? β
βββββββββββββ¬ββββββββββββββ
β
βββββββββββββββββΌββββββββββββββββ
βΌ βΌ βΌ
βββββββββββββ βββββββββββββ βββββββββββββ
β Small β β Medium β β Large β
β (<10 docs)β β (10-1000) β β (1000+) β
βββββββ¬ββββββ βββββββ¬ββββββ βββββββ¬ββββββ
β β β
βΌ β βΌ
βββββββββββββββββ β βββββββββββββββββ
β SEMANTIC β β β RAG β
β Full-doc in β β β Required β
β context β β β β
βββββββββββββββββ β βββββββββββββββββ
β
ββββββββββββββ΄βββββββββββββ
βΌ βΌ
βββββββββββββββββ βββββββββββββββββ
β Query-based? β β Synthesis? β
β βββββββββββββ β β βββββββββββββ β
β RAG β β Hybrid β
β (retrieval) β β RAG + Summary β
βββββββββββββββββ βββββββββββββββββ
Approach Comparison
| Factor | Prompting | RAG | Semantic | Fine-Tuning |
|---|---|---|---|---|
| Setup time | Minutes | Days | Hours | Weeks |
| Knowledge size | <10K tokens | Unlimited | <200K tokens | In weights |
| Update speed | Instant | Minutes | Instant | Hours |
| Cost per query | Medium | Medium-High | High | Low |
| Accuracy | Variable | High | High | Highest |
| Coherence | High | Medium | Highest | High |
| Citations | Manual | Automatic | Manual | None |
1. Prompting (In-Context Learning)
When to Use
- Rapidly changing requirements
- Small, stable knowledge sets
- Flexibility > consistency
- Prototyping and testing
Catalog Applications
- Email tone/style guidance
- Output format specifications
- Role-specific instructions
- Task-specific templates
Example
STYLE_PROMPT = """
You write sales emails with these characteristics:
- Opening: Specific observation about prospect
- Length: Under 150 words
- CTA: Single, low-friction ask
- Tone: Professional but human
- Avoid: "I hope this finds you well", exclamation points
"""
Limitations
- Consumes context window
- No dynamic updates without code changes
- Can't handle large knowledge bases
2. RAG (Retrieval-Augmented Generation)
See ENT-RAG for core concepts.
When to Use
- Knowledge base > 50K tokens
- Frequent knowledge updates
- Need source citations
- Q&A over documents
- Domain-specific accuracy critical
Architecture
Query β Embed β Vector Search β Top-K Chunks β LLM β Response
Best Practices
| Practice | Why | Implementation |
|---|---|---|
| Chunk wisely | Bad chunks = bad retrieval | 500-1000 tokens, 10-20% overlap |
| Embed metadata | Context improves ranking | Include title, section, date |
| Hybrid search | Keywords + semantics | BM25 + vector, reciprocal rank fusion |
| Rerank results | Improve precision | Cross-encoder (Cohere, BGE) |
| Cite sources | Trust + audit trail | Return chunk IDs with response |
| Handle "not found" | Avoid hallucination | Explicit fallback path |
Common Failures
| Failure | Cause | Fix |
|---|---|---|
| Wrong chunks | Poor embedding/chunking | Hybrid search, better chunks |
| Hallucination | LLM fills gaps | Stricter prompts, citations |
| Missed info | Top-K too low | Increase K, reranking |
| Incoherent | Chunks lack context | Better boundaries |
Catalog Applications
| Agent | RAG Application |
|---|---|
| Policy Q&A | Answer from policy docs |
| Incident Analysis | Similar past incidents |
| Competitive Intel | Search competitor history |
| Account Health | Similar account patterns |
3. Semantic (Full-Document Context)
When to Use
- Document < 50K tokens
- Task requires document structure understanding
- Cross-references within document matter
- "Read the whole thing" is correct approach
Advantages Over RAG
| Factor | RAG | Semantic |
|---|---|---|
| Coherence | Chunks may lack context | Full context |
| Structure | Lost in chunking | Preserved |
| Cross-refs | May miss | Captured |
| Setup | Complex | Simple |
Token Management
def smart_context_builder(data, max_tokens=100000):
"""Prioritize context when approaching limits."""
context_parts = []
token_count = 0
# Priority 1: Recent/critical (always include)
critical = get_critical_data(data)
context_parts.append(critical)
token_count += count_tokens(critical)
# Priority 2: Supporting data (include if room)
if token_count < max_tokens * 0.7:
supporting = get_supporting_data(data)
context_parts.append(supporting)
# Priority 3: Historical (summarize if needed)
remaining = max_tokens - token_count
history = get_history(data)
if count_tokens(history) < remaining:
context_parts.append(history)
else:
context_parts.append(summarize(history, remaining))
return combine(context_parts)
Catalog Applications
| Agent | Semantic Application |
|---|---|
| QBR Prep | Full account history |
| Proposal Generator | Full RFP document |
| Meeting Prep | Complete CRM record |
| Contract Review | Full agreement text |
4. Fine-Tuning
See ENT-FINE-TUNING for core concepts.
When to Use
- Consistent output format required
- Latency critical (<500ms)
- High volume (cost matters)
- 500+ examples available
- Task is narrow and stable
Not Suitable When
- Requirements change frequently
- Need current/dynamic knowledge
- <100 examples
- Need reasoning transparency
Catalog Applications
| Agent | Fine-Tuning Use |
|---|---|
| CRM Extractor | Email β structured fields |
| Expense Categorizer | Transaction β category |
| Resume Screener | Resume β assessment |
Hybrid Architectures
RAG + Semantic Summarization
Query β RAG Retrieval β Chunks β Summarizer β Coherent Context β Generator
Use when: Need both retrieval precision and narrative coherence
Fine-Tuned Extractor + RAG Validator
Document β Fine-Tuned Extractor β Fields β RAG Policy Check β Validated Output
Use when: Need consistent extraction with policy compliance
Tiered Knowledge
βββββββββββββββββββ
Hot (prompting) ββββΊβ Style, format β <100 tokens
βββββββββββββββββββ€
Warm (semantic) ββββΊβ Current context β <50K tokens
βββββββββββββββββββ€
Cold (RAG) βββββββββΊβ Full knowledge β Unlimited
βββββββββββββββββββ
Selection Checklist
## Knowledge Architecture Decision
### Requirements
- [ ] Knowledge size: _____ (tokens/documents)
- [ ] Update frequency: _____ (realtime/daily/weekly/rarely)
- [ ] Query type: Search / Synthesis / Extraction
- [ ] Accuracy requirement: Critical / Important / Nice-to-have
- [ ] Citations needed: Yes / No
- [ ] Budget per query: $_____
### Recommendation
Based on above:
- [ ] Prompting (small, static, flexible)
- [ ] RAG (large, searchable, citable)
- [ ] Semantic (medium, coherent, structured)
- [ ] Fine-tuning (consistent, fast, high-volume)
- [ ] Hybrid: _____
### Implementation Path
See: [Implementation Guide: RAG Systems](/docs/MPD-TEC-IMPL-RAG-01.html) or relevant guide
Related Documents
- ENT-RAG - RAG concept definition
- ENT-FINE-TUNING - Fine-tuning concept definition
- Pattern: Learning Strategies (Prompting to RLHF) - Learning strategy selection
- Implementation Guide: RAG Systems - RAG implementation guide