Architectural Traits of AI

Pillar 3 of 3

Integration Patterns & Paradigms

Choosing the model is half the job — the other half is wiring it into the enterprise. Two decisions dominate: how the model gets access to your data, securely (RAG), and how much you invest in shaping its behavior (prompting → RAG → fine-tuning, in that order).

RAG — Retrieval-Augmented Generation

RAG connects an LLM to enterprise data without retraining it. Documents are chunked, embedded, and indexed in a vector database; at question time the system retrieves the most relevant chunks and hands them to the LLM inside the prompt. The model answers from what it was shown — not from what it vaguely remembers.

RAG reference architecture

OFFLINE — keep the index fresh ONLINE — every user question Enterprise data wiki · DB · SharePoint · tickets Chunk + embed embedding model → vectors Vector database index + document ACLs User question "What's our refund policy?" Embed query → retrieve top-k only chunks this user may see similar chunks LLM prompt = question + chunks Grounded answer + citations traceable to the source documents
Security lives in the retrieval step: enforce document ACLs when fetching chunks, and the model can only ever leak what the asking user was already allowed to read.

Why architects love it

  • Fresh knowledge: update the index, not the model
  • Citations → auditable, trust-building answers
  • Access control enforced at retrieval time
  • No training cost or MLOps pipeline needed

Failure modes to design for

  • Answer quality is capped by retrieval quality
  • Chunking strategy makes or breaks it
  • Context window budgets how many chunks fit
  • Stale index = confidently outdated answers

Typical uses

  • Support assistant over product documentation
  • Policy / legal Q&A with source citations
  • Internal "ask our wiki" for employees

Prompt engineering vs RAG vs fine-tuning

These are not competitors — they are an escalation ladder. Each rung buys more capability at roughly ten times the operational burden. Walk up only when the cheaper rung demonstrably fails.

The escalation ladder — decide in this order

New LLM requirement Can clear instructions + a few examples in the prompt get you there? yes Prompt engineering hours of work · cheapest · always start here no Is the gap missing knowledge — your documents, fresh or private facts? yes RAG days–weeks · adds retrieval infra · citable no Is the gap behavior — style, format, domain jargon, or unit cost at very high volume? yes Fine-tuning weeks + MLOps · retrain per base model no → rethink the requirement — you may need a different model family (pillar 1)
Fine-tuning is not how you add knowledge — models fine-tuned on facts still hallucinate them. Knowledge is RAG's job; fine-tuning shapes behavior.
ApproachEffort & timeCost profileBest forRisks
Prompt engineering Hours–days, no ML team Inference tokens only Instructions, output format, tone, few-shot classification Brittle prompts; behavior shifts between model versions — keep an eval set
RAG Days–weeks, app team + data pipeline Vector DB + ingestion infra + tokens (retrieved chunks bill as input) Company knowledge, fresh facts, auditable answers with citations Retrieval quality, chunking strategy, stale index, context budget
Fine-tuning Weeks; needs curated training data + MLOps Training runs + dedicated hosting; redo when the base model upgrades Consistent style/format, domain language, making a small cheap model match a big one at volume Overfitting, catastrophic forgetting, doesn't reliably add facts, ongoing maintenance
Cost-effectiveness rule: fine-tune only when (a) prompting + RAG demonstrably can't hit the quality bar, or (b) volume is so high that a fine-tuned small model beats a prompted large model on unit economics. Everything else is premature optimization with an MLOps tax attached.
← Pillar 2: Operational MetricsContext, latency, VRAM, evaluation