Reference Architecture Diagram + Narrative (feature→model→inference with policy-by-design)
┌──────────────────────────────────────────────────┐
│ DATA SOURCES │
Lakehouse (Bronze/Silver/Gold) │ Streams (Kafka) │ OT/IoT │ SaaS/CRM/ERP │
└──────────────┬───────────────┬────────┬──────────┘
│ │ │
▼ ▼ ▼
┌───────────────────────────────────────────────────────────────────┐
│ DATA & FEATURE ENGINEERING LAYER │
│ • ETL/ELT (Spark/DBT) • Streaming transforms • Quality/Contracts │
│ • Feature Store (offline+online) • Lineage & Governance │
│ • PII/PHI/PCI policies: tokenize/mask/pseudonymize (KMS/DLP) │
└───────────────┬───────────────────────────────────────────────────┘
│ versioned features (IDs, TTLs)
▼
┌───────────────────────────────────────────────────────────────────┐
│ TRAINING / EVALUATION FABRIC │
│ • GPU/CPU/HPC clusters (on-prem/cloud) • Orchestrators (K8s) │
│ • Model Registry (versions, metadata, cards) • Dataset registry │
│ • Eval pipelines: perf, bias, safety, privacy, cost, carbon │
│ • Prompt/test corpora for LLMs (red-teams & safety suites) │
└───────────────┬───────────────────────────────────────────────────┘
│ signed artifacts (models, SBOMs)
▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ INFERENCE & APPLICATION FABRIC (ONLINE) │
│ • API Gateway / Service Mesh (mTLS) • Canary/blue-green • A/B switches │
│ • Online Feature Store (low-latency) • Vector DB (RAG) • Caching │
│ • Prompt Tooling: templates, routers, rewriters, function-calling │
└───────────────┬───────────────────────────────────────┬────────────────────┘
│ │
▼ ▼
┌──────────────────────────────────────┐ ┌──────────────────────────────────┐
│ LLM GUARDRAIL & SAFETY LAYER │ │ DOMAIN APPS & CHANNELS │
│ • Policy-as-code (OPA) for prompts │ │ • Assistants/Agents • CCaaS/UC │
│ • Content filters (tox/PII/safety) │ │ • Web/Mobile • Backoffice APIs │
│ • Retrieval allow-lists (RAG scope) │ │ • OT/IoT actions via brokers │
│ • Redaction & watermarking │ └──────────────────────────────────┘
└───────────────┬────────────────────────────────────────────────────────────┘
│
▼
┌────────────────────────────────────────────────────────────────────┐
│ OPERATIONS, TRUST & COST (CROSS-CUTTING) │
│ • IdP/MFA • PAM/JIT • HSM/KMS/PKI • SIEM/SOAR • AIOps (latency) │
│ • ITSM/CMDB • GRC/WORM (audit) • FinOps/Carbon (per-call cost/CO2)│
└────────────────────────────────────────────────────────────────────┘
Narrative (how AI stays useful, lawful, and affordable)
1) Purpose & posture
- Objective: Deliver a governed AI/ML platform that can train and serve classical models and LLMs—safely, privately, and cost-transparently—across all prior architectures.
- Posture: Data-centric policies from the start, signed artifacts, guardrails at inference, and FinOps visibility per model and per request.
2) Data & features (syntax of learning)
- ETL/ELT + stream transforms produce contracted, quality-checked datasets.
- Feature Store provides offline (training) and online (inference) slices with feature IDs/TTL/lineage.
- Policy/DLP/KMS enforces tokenization/masking/pseudonymization before any feature is exposed.
3) Training & evaluation (semantics of models)
- Runs on GPU/CPU/HPC (on-prem/cloud) via K8s.
- Model Registry holds versions, SBOMs, and model cards.
- Evaluation pipelines score accuracy, latency, bias/fairness, safety, privacy leakage, cost, and carbon; failing gates block promotion.
4) Inference fabric (meaning in motion)
- API gateway + service mesh (mTLS) with canary/blue-green and A/B.
- RAG via vector DB constrained by retrieval allow-lists (only sanctioned corpora).
- Prompt ops: templating, routing, re-writing, tool/function calling; cache hot prompts.
5) Guardrails & safety (law at the boundary)
- Policy-as-code (OPA) checks every request/response: domain, role, residency.
- Content filters for toxicity/PII/safety; redaction pre/post; watermarking for provenance.
- Kill-switches: per-tenant rate limiting, domain off-ramps, and escalation to human.
6) Operations & truth (pragmatics)
- IdP/MFA + PAM/JIT lock down consoles and model endpoints; HSM/KMS/PKI protect keys.
- AIOps watches latency/SLO, drift, and error spikes; SIEM/SOAR correlates misuse; ITSM/CMDB ties incidents to versions.
- FinOps/Carbon attaches $ and CO₂ per call, per model, per tenant—actionable budgets.
7) Resilience patterns
- Regional replicas for models & vector stores; hot-warm failover; prompt-cache fallback.
- Safe degrade: fall back to smaller models or retrieval-only answers if GPU capacity is constrained.
8) Reference KPIs
- P50/P95 inference latency: <300 ms / <900 ms (domain-tuned)
- Guardrail block precision/recall: ≥95%/≥98% on policy test suite
- RAG faithfulness score: ≥0.9 on eval set; hallucination rate: <1% on gated routes
- Cost/CO₂ variance to budget: ≤5% monthly
- Security incident MTTR (endpoint): ≤1 h
9) Minimal BOM (aligned with the stack)
ETL/ELT (Spark/DBT), Feature Store (offline/online), Dataset/Model Registry, GPU/CPU/K8s training farms, Eval pipelines & safety suites, Inference gateway + service mesh, Vector DB (RAG), Prompt tooling, OPA policy engine, Content filters/redaction/watermarking, IdP/MFA + PAM/JIT, HSM/KMS/PKI, SIEM/SOAR, AIOps/Observability, ITSM/CMDB, FinOps/Carbon, GRC + WORM audit.