How do teams keep agents reliable and governed: harnesses, verifiers, evals, and human review?

To keep AI agents reliable and governed, teams move beyond simple demos to implement a multi-layered "safety and readiness framework" involving quantitative evaluations, deterministic and LLM-based verifiers, and risk-adjusted human oversight.

The Foundation: Quantitative Evaluations (Evals)

Creators emphasize that building evals must precede building the technology itself. Without quantitative metrics to define "good enough," developers are merely "iterating on a demo" rather than building a product Y Combinator — Waymo Co-CEO Dmitri Dolgov: The Demo Is Only 1% Of The Work @ 42:49.
* Defining "Good": Governance begins with domain expertise to define exactly what a successful task looks like and what specific information a model must extract Y Combinator — From Idea to $650M Exit: Lessons in Building AI Startups @ 15:15.
* The Liveness Balance: There is a constant tension between "liveliness" (keeping work moving) and "verification" (assurance of correctness). Teams must avoid "verification theater," where high volumes of agent output lead humans to sign off on tasks they haven't actually reviewed AI Engineer — What Does Done Even Mean? Agents and Paperclip's Liveness Model - Dotta, Paperclip @ 00:00.

Verifiers and Harnesses

Teams use "verifiers" to analyze the final state, trace, and artifacts of an agent's work. These fall into two main categories:
* Deterministic Verifiers: High-reliability, low-flexibility checks similar to traditional unit tests. These are ideal for checking tool calls, API responses, and database states AI Engineer — From Agent Traces to Agent Simulations — Rustem Feyzkhanov, Snorkel AI @ 09:12.
* LLM Judges: High-flexibility, low-reliability checks where a second LLM evaluates the reasoning or "taste" of the first model. While flexible, these can produce false positives by over-rewarding plausible but incorrect solutions Y Combinator — Going In Deep On Data | YC Paper Club @ 21:21.
* Agent Harnesses: For long-running autonomous agents, Nick Saraev recommends "harnesses" that enforce explicit rules: confirming API calls above a cost threshold, forbidding the modification of credentials, and logging all self-modifications as a change log Nick Saraev — AGENTIC WORKFLOWS: Build & Sell AI Automations (2026) @ 3:52:02.

Governance: Human-in-the-Loop (HITL) and Transparency

Governance strategies often follow the "VAULT" framework (Verify, Augment, Loop humans in, Transparency).
* Risk-Adjusted Oversight: Human checkpoints should scale with the consequence of the action. Lower-risk tasks (like organizing notes) can run autonomously, while higher-risk actions (like messaging clients or spending money) require an approval step Nate Herk | AI Automation — Everything Goldman Sachs Taught Me About AI (In 10 minutes) @ 06:04.
* Draft-First Workflow: A common safety pattern is setting automations to "draft, not send." This allows agents to prepare work (e.g., email drafts) while leaving the final deployment to a human Nate Herk | AI Automation — Everything Goldman Sachs Taught Me About AI (In 10 minutes) @ 06:04.
* Transparency and Auditing: To maintain trust, systems must be transparent. Every execution should be logged so that managers or regulators can understand how a result was reached Nate Herk | AI Automation — Everything Goldman Sachs Taught Me About AI (In 10 minutes) @ 06:04. Some suggest that future "AI auditing systems" could provide unbiased, memory-less audits of company actions to ensure they abide by mission statements Y Combinator — Ask These Questions Before Starting An AI Startup @ 15:16.

— Sources: 10 videos across 4 creators