AI & Cognitive Systems
Models that route attention and decisions. Document data, test outputs, and require human override in safety‑critical contexts.
Healthy uses
- Retrieval‑augmented assistants with citations for operators.
- Forecasting/optimization with auditable constraints.
- Content ranking with visible feedback controls.
Risks to avoid
- Opaque models affecting safety/finance without oversight.
- Prompt/data leakage; undisclosed synthetic media.
- Misaligned KPIs that reward engagement over truth.
Controls & guardrails
- Model cards, data sheets, evaluation suites as deploy gates.
- Human‑in‑the‑loop for high‑impact actions; override paths.
- PII scrubbing; allowlists; content safety filters.
- Adversarial/red‑team tests; rollback and kill‑switch.
- Logging with trace IDs; decision provenance retained.
Signals & metrics
- Grounded‑answer rate
- Evaluation pass rate
- Override frequency
- Incident count
Tech hooks
AI/ML (RAG + eval), Security (OIDC, KMS, SBOM, signing), Observability (OpenTelemetry), Data (catalog/lineage).
Related playbooks
Point A → Point B, Fast
Architecture Sprint (1–2 weeks): Current → Target → 1–90-day plan. D1 maps, D2 target, D3 plan, D4 exec brief.