Agentic AI Engineering · Module 15 of 16

Production Architecture

A production agent is not one model call. It is a software system with an API, state store, model/runtime, tool gateway, authorization, approvals, observability, evaluation and recovery boundaries.

Engineering focus: Production agent architecture, boundaries and operating model

In plain English

What problem are we solving?

A production agent is not one model call. It is a software system with an API, state store, model/runtime, tool gateway, authorization, approvals, observability, evaluation and recovery boundaries.

Framework names come later. First understand what the application is doing and where its control boundaries belong.

Running case · Operations Investigator

Across the course, imagine an assistant that investigates operational problems using narrow tools. The design progressively adds plans, specialist agents, approvals, reliability controls and MCP while keeping model output separate from execution authority.

Learning objectives

By the end of this module, you should be able to:

Place agent components into clear trust and failure boundaries.
Separate model/runtime responsibilities from tool gateway, policy, state and observability.
Design rollout, fallback and rollback controls for an agent service.

Separate control plane from capability plane

The agent runtime decides what action to propose. A tool/MCP gateway mediates external capabilities. Policy and authorization should not be embedded only in prompts. State and audit storage remain authoritative application services.

Design for least privilege and network boundaries

Place tools behind authenticated services, use workload identities, restrict outbound destinations, isolate high-risk capabilities and avoid giving the model host direct access to every production network.

Make release artifacts reproducible

A release record should identify model/runtime, prompts/instructions, tool schemas, policies, MCP server versions, evaluation dataset/results, dependencies and deployment configuration. Without this, rollback and incident diagnosis become guesswork.

Operate with progressive delivery

Use staging, shadow/dry-run modes, canaries, low-risk cohorts, feature flags and kill switches. Promotion depends on evaluation plus production telemetry. Rollback should restore a known-good runtime/policy/tool configuration, not only a model name.

Worked examples

Reference architecture

User/API
  ↓
Agent service ── state store
  ↓
Policy + authorization
  ↓
Tool/MCP gateway ── approvals
  ↓
External systems

All layers → traces/metrics/audit
Release pipeline → eval gates → staged rollout

Each layer has a distinct responsibility.
External side effects cross explicit policy and authorization boundaries.

Deployment decision

candidate=A17
eval=PASS
critical_policy_violations=0
canary_success=99.4%
p95=2.1s
budget_errors=0
→ promote 25% → observe → promote/hold

Evaluation and runtime evidence both matter.
Promotion is incremental and reversible.

Try it · core lab · account-free

Validate a production agent architecture

Goal: Check a simplified architecture for missing security, state, observability, recovery and release controls.

Before you start

Python 3 only.

STEP 1

Run the architecture validator

python3 public/labs/agentic-ai/15_production_architecture.py

STEP 2

Review the unsafe architecture

The first fixture intentionally omits authorization, audit and durable state.

STEP 3

Validate the production-shaped architecture

The second fixture must satisfy all required boundaries and rollout controls.

Download this Python lab Download all current labs

Verify

The unsafe design is rejected with specific missing controls.
The complete design passes all architecture rules.

Expected outcome

You can explain why “LLM + tools” is not a production architecture.
You can point to where authorization, state, approvals, telemetry and rollback live.

Break / fix · diagnose

If the validator accepts the unsafe design, add hard requirements for external side effects and durable evidence.
If responsibilities overlap, assign one authoritative owner for policy, state and execution.

Production view

Production readiness is largely about explicit ownership and boundaries: who decides, who authorizes, who executes, what is durable, what is observable, and how the system stops or rolls back.

Avoid these traps

Common mistakes

Putting authorization in the prompt instead of the tool boundary.
Using one database/log as an undefined catch-all for state and audit.
Deploying without a kill switch or rollback target.
Treating model version alone as the complete release identity.

Knowledge check

Explain the control decision, not just the vocabulary.

Open every answer only after committing to your own answer. Opening all four answers records knowledge-check activity in My Learning.

1Where should external side-effect authorization normally be enforced?

At the application/tool gateway or service boundary using authenticated identity and policy, before execution.

2Why keep durable state outside the model context?

Application state must survive restarts, support reconciliation and remain authoritative independent of what the model currently sees.

3What should an Agentic AI release identity include besides the model?

Instructions/configuration, tool schemas, policies, MCP/server dependencies, evaluation dataset/results and deployment/runtime configuration.

4Why use staged rollout for an agent?

It limits blast radius and lets production telemetry validate the candidate before broad exposure.

My Learning

Record your progress for this module.

Progress is saved in this browser immediately. When you sign in to My Learning, it can also sync across devices.

Not started

Core lab attemptedRecord this after meaningfully completing or attempting the module's core lab.Knowledge checks attemptedThis is marked automatically after you open every answer in the module; you can also record it manually.

Attempt the core lab and knowledge checks before marking the module complete.