When AI systems were just models behind APIs, life was comparatively simple: train, deploy, tune. That world is gone. Modern AI looks less like a single engine and more like a busy city — networks of small, specialised agents that call APIs, automate workflows and make decisions faster than humans can watch. A recent O’Reilly Radar essay, “AI Agents Need Guardrails,” argues that the binding constraint on these systems is no longer performance. It is governance.
From pipeline to “small brain network”
A single customer-service agent might call a summarisation agent, which queries a retrieval agent, which hits an internal API — asynchronously, across systems. Debugging stops being “check the log file.” When an agent sends confidential data to the wrong API, someone has to reconstruct the whole story: which agent called whom, what data went where, and why each decision was made. That requires tracing lineage, context and intent across the entire ecosystem, not just collecting logs.
Governance as the missing runtime layer
Governance frameworks are not new — the NIST AI Risk Management Framework and the EU AI Act define principles such as transparency, accountability and human oversight. The essay’s point is that principles on paper do not enforce themselves at runtime. What is missing is a governance layer that sits beside the agents while they run: checking that every action follows approved paths, every dataset stays where it belongs, and every decision can be traced afterwards.
Policy as code
The proposed mechanism is familiar from infrastructure engineering: policy as code. With a tool like Open Policy Agent (OPA), rules become version-controlled code — reviewable, testable and enforceable. Rules can express which agents may access sensitive datasets, which API calls require human review, and when a workflow must halt because risk is too high. Developers and compliance staff stop talking past each other; they work in the same repository. A Dockerized OPA instance can run inside the same Kubernetes cluster as the agents, quietly checking requests and stopping risky actions before they reach an API or data store. Governance stops being an abstraction and becomes another microservice: scalable, observable, testable.
Observability with semantic context
Standard telemetry — OpenTelemetry, Prometheus, Grafana Loki — already captures structured logs and traces. What is missing is semantic context: linking actions to intention and policy. The difference between logging “API call” and logging “agent FinanceBot requested API X under policy Y with risk score 0.7” is the difference between telemetry and governance. In Kubernetes, sidecar containers can inject this metadata into every request automatically.
Dynamic risk scoring
Not every action deserves the same scrutiny. A “summarise report” request is low risk; “transfer funds” or “delete records” is high risk. Assigning each task a dynamic risk score — computed from the agent’s role, data sensitivity and confidence level — lets the system decide in real time whether to allow an action automatically, require additional verification, or escalate to a human reviewer. Cloud tooling is heading the same way; Google Cloud’s Vertex AI model monitoring already supports risk tagging and drift detection, ideas that extend naturally to agent actions. The goal is not slower agents but context-aware ones.
| governance theory | engineering implementation |
| transparency | Agent Activity Log, Explainability Metadata |
| Accountability | Immutable Audit Trails in Cloud Logging/Chronicle |
| strength | Canary testing, rollout control in Kubernetes |
| risk management | Real-time scoring, human-in-the-loop review |
Regulation as an engineering blueprint
Read this way, NIST AI RMF and the EU AI Act are not just legal mandates — they double as engineering blueprints. Mapping their requirements onto cloud and container tools turns compliance into configuration: developers own agent logic, security teams manage policy rules, and compliance officers watch dashboards instead of combing raw logs.
Lessons from the field
The essay’s author reports three lessons from building governance into multi-agent pipelines. First, smarter controls beat more controls: requiring manual approval for everything paralyses agents, so the low-risk 90% should be automated. Second, logging everything is not enough — governance needs explanation logs with correlation IDs and metadata that map events back to business rules. Third, governance must be part of the developer experience: if compliance feels like a gatekeeper, developers route around it; if it feels like a built-in service, they use it willingly. In one financial-technology deployment, policies were enforced at the Kubernetes ingress layer, before requests ever reached the agent pods.
Governance theater vs. governance engineering
The essay draws a sharp line between governance theater — policies written but never enforced — and governance engineering — policies compiled into running code. Theater produces binders. Engineering produces metrics: the percentage of agent actions logged, policy violations caught before execution, average human review time for high-risk tasks. What can be measured can be improved; that is how an organisation moves from claiming its systems are safe to demonstrating it.
Why this matters beyond big tech
The pattern scales down. Even a small firm running a handful of AI automations faces the same questions in miniature — which data an assistant may touch, which actions need a human sign-off, and what record exists when something goes wrong. That gap between adopting agents and governing them is exactly why most AI agent projects stall, and it is why enterprise platforms now sell governance as a headline feature, as seen in Databricks’ governed MCP agent workflows.
Limitations and what to watch
Policy-as-code is necessary but not sufficient: OPA can only enforce rules someone thought to write, risk scores are heuristics that can be gamed or miscalibrated, and semantic logging adds engineering cost that small teams may defer. Regulatory interpretation of the EU AI Act’s requirements for agentic systems is still evolving. Worth watching: standardised governance interfaces for agent frameworks, and whether “governance checks” become as routine in CI/CD pipelines as automated tests — the essay’s closing prediction.
