Agents keep changing their answers. Harness just built delivery pipelines that don’t care. (The New Stack)

Adrian Bridgwater, The New Stack, 22 Jul 2026, reporting Harness‘s launch of AI Agent Development Lifecycle (DLC) and interviewing trevor-stuart (SVP/GM). The pitch: ship agents through the same pipelines, gates and approvals that already govern application code.

Framing stat, from the 2026 Gartner CIO and Technology Executive Survey: only 17% of organizations have deployed AI agents. Stuart’s diagnosis of the other 83% — an agent that works in a demo but can’t be trusted in production gets left in the sandbox, and the result is “technically a success, but practically useless.”

The argument

Application code is deterministic, so the same test on the same code gives the same answer twice. An agent’s model “decides how to complete a task, so the same agent, given the same input, can choose a different tool or take a different action from one run to the next.” Two consequences the article draws out: a passing test “offers no guarantee it will pass the next time,” and “incidents stop being reproducible on demand” — which takes the standard debugging playbook with it.

The move is to stop trying to fix the agent:

“Don’t think about trying to make the agent predictable; instead, make the pipeline around it predictable. … grade the response on correctness, safety, and performance, and wire that score straight into the pipeline as a pass-fail gate.” — trevor-stuart

Stuart concedes the limit openly: agents and their output are not going to be reproducible any time soon. “What we can make reproducible is the record of what happened: every model call, every tool call, every step it took, all captured.” Determinism moves from the artifact to the process and the record — see agent-evals for the eval-score-as-gate mechanism this rests on.

The security half: the attack surface is “larger and dynamic in nature” because agents connect to tools and APIs, spawn sub-agents, and inherit trust from every model they touch. Static scans were not built for that.

What shipped

Five pieces, spanning test → deploy → operate → govern:

• AI Evals — eval datasets, scoring functions, and quality gates that catch regressions whenever the agent or the model underneath it changes.
• Agent deployments — the canary releases, approvals and OPA guardrails Harness already applies to Kubernetes deployments, pointed at managed agent runtimes.
• AI configs — releasing and managing prompts and model changes at runtime.
• AI asset catalog — discovers every agent, skill and plugin across an org’s repositories and attaches an owner to each.
• AgentTrace — records a single agent run and a full multi-step session: which path the agent took, where it slowed down, how different models or prompts change the outcome.

The tracing primitives are being open-sourced as harness-sdk and harness-evals. This extends Harness’s June 2026 Autonomous Worker Agents (agents running as governed steps inside delivery pipelines) from a pipeline step to the whole agent lifecycle.

On ownership, Stuart rejects a central gatekeeper: “There’s no single governance role sitting on top of everything. Ownership attaches the moment something gets built, and it stays traceable back to whoever’s responsible.”

What it doesn’t show

Asked for the effectiveness trade-off against ad-hoc agent piloting, Stuart declines: “There’s no perfect stat here, so I won’t pretend to give you a clean dollar figure.” What he offers instead is from Harness’s own 2026 State of Engineering Excellence report — 31% of a developer’s day goes to AI-related work “that shows up in no metric at all,” and 94% of engineering leaders say things like tech debt, validation time and burnout are missing from what they track. That is a measurement-gap argument, not evidence the product closes it.

Tier

T3 — trade press, but every claim traces to one vendor’s launch and one vendor executive; the supporting survey is also Harness’s own. No independent measurement, no customer, no benchmark. The problem framing (nondeterminism breaks test-and-incident reproducibility; gate on graded output instead) is the durable part; the five-product answer is one vendor’s bet.

Related

agent-evals · harness-io · trevor-stuart · agent-loops-verification · agent-guardrails · agentic-coding-harness