signal insight

Anthropic adds a plugin eval command to Claude Code for scored, reproducible plug…

Claude Code 2.1.269, released September 11, 2026, introduced the claude plugin eval command, which runs a plugin against realistic prompts, grades the output with pass/fail graders, and compares results against a no-plugin baseline. It produces shareable HTML reports and CI-friendly exit codes, turning plugin and skill quality into measurable numbers.

Published Sep 11, 2026 Updated Sep 15, 2026 2 sources

AnthropicClaude Codeai coding toolsfeature launchmedium impact

ai-coding-toolsevalspluginsskillscideveloper-toolsfeature launch

Impact: medium
Confidence: 80%
Change type: feature launch
First seen: Sep 11, 2026
Last updated: Sep 15, 2026
Audience: developersplugin-authorsplatform-engineers
Status: Ready

Summary

What changed

On September 11, 2026, Claude Code v2.1.269 shipped the claude plugin eval and claude plugin eval init commands, running realistic prompts through multiple graders with three runs per case and a no-plugin baseline, plus an HTML report, cost ceilings, concurrency controls, trust settings, and exit codes built for CI.

Why it matters

Plugin and skill ecosystems have lacked objective quality measurement; a built-in eval harness with a no-plugin baseline lets developers prove a plugin actually improves outcomes and gate merges in CI, raising the reliability bar for the growing Claude Code extension ecosystem.

Evidence excerpt

Claude Code v2.1.269 (September 11, 2026) added the claude plugin eval command: realistic prompts, pass/fail graders, three runs per case, a no-plugin baseline that exposes whether a plugin does anything, an HTML report, and exit codes built for CI.