Summary
Around September 10, 2026, LangChain and Harbor detailed a unified evaluation stack that runs each agent trial in an isolated LangSmith sandbox and streams results into LangSmith observability with full agent traces. Teams can run hundreds of trials in parallel and rerun evaluations when skills, models, or tool descriptions change, turning agent evaluation into regression testing.
What changed
LangChain and Harbor integrated Harbor agent evaluation with LangSmith sandboxes and observability, giving each trial a fresh isolated sandbox, parallel execution, and automatic reruns when skills, models, or tool descriptions change.
Why it matters
Reliable regression testing for stateful agents is a persistent gap as teams move agents to production; attaching verifier rewards and traces to every trial makes behavior changes diagnosable rather than mysterious. It strengthens LangSmith as an evaluation and observability control plane for agent teams.
Evidence excerpt
Each trial gets a fresh LangSmith sandbox so trials never share state, and with the langsmith plugin every job lands as a dataset and experiment with the verifier reward as feedback and agent traces attached.