Gnu Group Learning AcademyGnu Group ↗
AI Engineering Fundamentals · Module 04 of 12

Get predictable answers and safe tool requests

Move beyond free-form text. Define the shape of an answer, validate it before software trusts it, and treat a model’s request to use a tool as a proposal that still needs normal application authorization.

Engineering focus: Structured outputs and tool contracts

In plain English

What problem are we solving?

Move beyond free-form text. Define the shape of an answer, validate it before software trusts it, and treat a model’s request to use a tool as a proposal that still needs normal application authorization.

The rest of this module introduces the formal terminology and production controls behind that idea. You are not expected to memorize the vocabulary before using it.

Running case · Atlas Assist

Carry the same support assistant forward: approved KB, source citations, explicit insufficient-evidence fallback and one bounded read-only asset lookup. Reuse prior artifacts so each module adds evidence to one production AI system.

Learning objectives

By the end of this module, you should be able to:

  • Define a structured output schema and validate model output against it.
  • Design tool definitions with narrow arguments and explicit authorization outside the model.
  • Explain why tool calling is a proposal, not permission to perform a side effect.

Structured output is an interface contract

If downstream code needs fields, types or enumerated states, define them explicitly. Valid JSON alone is weaker than schema-conforming output: keys, types and allowed values matter.

Validation still belongs in application code

Schema-constrained generation improves reliability but your application still handles refusals, truncation, unavailable tools, business validation and authorization. Never deserialize model output directly into a privileged action without checks.

A tool call is a request from the model

The application decides whether the requested tool exists, whether the caller is authorized, whether arguments are valid and whether confirmation is required. The model must not be the security boundary.

Minimize tool authority

Prefer narrow read-only functions over generic shell/SQL/network tools. Side-effecting tools should be idempotent where possible and carry explicit scopes, limits and audit records.

Separate planning from execution

For high-impact actions, generate a proposed plan or structured change first, validate it, show it to a human or policy engine, and only then execute the bounded operation.

Worked examples

Response schema

{
  "type":"object",
  "required":["status","answer","source_ids"],
  "properties":{
    "status":{"enum":["ANSWERED","INSUFFICIENT_EVIDENCE"]},
    "answer":{"type":"string"},
    "source_ids":{"type":"array","items":{"type":"string"}}
  }
}
  • Schema-constrained output is easier to validate and route than free-form prose.

Tool security boundary

model proposes tool(name,args)
→ application validates schema
→ application checks user/tool policy
→ optional confirmation
→ executor runs bounded function
→ result returned as data
  • The model never grants itself authority.
Try it · core lab · account-free

Lab 04 — Validate structured output and a tool proposal

Goal: Build deterministic validators around model-shaped JSON.

Before you start

  • No paid API or cloud account is required for the core lab.
  • Core labs use Python 3 and local files only.
  • Running case: you are engineering 'Atlas Assist', an internal support assistant over a small approved knowledge base with one read-only inventory tool.
  • Carry requirements, eval cases, threat assumptions and operational evidence forward across modules instead of starting over.
STEP 1

Validate an answer contract

python3 - <<'PY2'
import json
obj={"status":"ANSWERED","answer":"Reset requires approval.","source_ids":["kb-17"]}
assert obj['status'] in {'ANSWERED','INSUFFICIENT_EVIDENCE'}
assert isinstance(obj['answer'],str) and isinstance(obj['source_ids'],list)
assert all(isinstance(x,str) for x in obj['source_ids'])
print('PASS: response contract')
PY2
STEP 2

Reject an over-broad tool argument

python3 - <<'PY2'
proposal={"name":"lookup_asset","arguments":{"asset_id":"*"}}
asset=proposal['arguments']['asset_id']
allowed=asset.startswith('srv-') and asset[4:].isdigit()
print('authorized?',allowed)
assert not allowed
print('PASS: tool proposal rejected before execution')
PY2
STEP 3

Design confirmation policy

Mark each tool in your design as read-only, reversible write, or irreversible/high-impact. State which classes require human confirmation.

Verify

  • Invalid business arguments are rejected even if JSON is syntactically valid.
  • Authorization is represented outside the tool description.

Expected outcome

  • A response validator plus a bounded tool proposal policy.

Break / fix · diagnose

  • If the model decides authorization, move the decision into trusted application code.
  • If one generic tool can execute arbitrary commands, redesign it into narrow functions.
Production view

Structured outputs and function tools turn model behaviour into software interfaces that can be validated, authorized and observed.

Local real-tool practicum · optional but recommended

Optional practicum — schema-constrained Ollama output

Ask a local model for a schema-constrained response and inspect the resulting JSON.

Prerequisites

  • Ollama installed and reachable on http://127.0.0.1:11434 for the real-tool path.

Resources created

  • Only local disposable files/processes used by the practicum.

Cost / risk

  • No cloud account required; model downloads consume disk and network bandwidth.
TOOL 1
MODEL=${MODEL:-qwen3:8b}; curl -s http://127.0.0.1:11434/api/chat -H 'Content-Type: application/json' -d "{\"model\":\"$MODEL\",\"messages\":[{\"role\":\"user\",\"content\":\"Classify this ticket: database timeout\"}],\"stream\":false,\"format\":{\"type\":\"object\",\"properties\":{\"category\":{\"type\":\"string\"},\"severity\":{\"type\":\"integer\"}},\"required\":[\"category\",\"severity\"]}}" | python3 -m json.tool | head -100

Verify

  • The model response content can be parsed as the requested structure.

Cleanup

  • Remove only the disposable files created for the lab; keep model caches only if you intend to reuse them.

Warnings

  • Schema validity does not prove factual correctness or authorization; those are separate gates.
Avoid these traps

Common mistakes

  • Using JSON mode when a concrete schema is required.
  • Executing model tool arguments without business validation.
  • Exposing generic privileged tools.
  • Assuming a schema removes the need for refusal/truncation handling.
Knowledge check

Can you connect the AI behaviour to a system control?

Concept and scenario checks. Reveal the answer only after you commit to a decision.

1What is the difference between JSON mode and structured output?

JSON mode targets syntactically valid JSON; structured output constrains the response to a defined schema.

2Who authorizes a model-requested tool call?

Trusted application policy, not the model.

3Why prefer narrow tools?

They reduce the blast radius and make validation, authorization and auditing tractable.

4Scenario: the model requests delete_account with valid JSON for a user who lacks permission. What should happen?

The application must deny the operation because schema validity is not authorization.

My Learning

Record your progress for this module.

Progress is saved in this browser immediately. When you sign in to My Learning, it can also sync across devices.

Not started

Attempt the core lab and knowledge checks before marking the module complete.