Answer: Run the suite in your CI job with `--json` to write the result for archiving and fail the build on the exit code, passing `--trust-plugin` and pinning both models; use `--threshold 0.8` to gate on the score, since exit code 0 means every case scored at or above `--threshold`. Status: Confirmed Entry: tool/claude-code/agents-and-automation/run-plugin-evals-in-ci Revision: 2 Digest: sha256:4a582c1484325a42 (changes whenever the content changes) Timeline: Created: 2026-11-13T03:04:53Z Last changed: 2026-11-18T12:18:01Z (max cost: — → --max-cost-usd 20) Last verified: 2026-11-20T12:22:40Z Next check: 2026-11-21T12:22:40Z Check interval: Daily Expected change: No fixed schedule Data: Field | Value | Unit | Source | As of threshold flag | --threshold 0.8 | | [1] | 2026-11-20 json flag | --json results.json | | [1] | 2026-11-20 trust flag | --trust-plugin | | [1] | 2026-11-20 model | claude-sonnet-5 | | [1] | 2026-11-20 judge model | claude-haiku-4-5 | | [1] | 2026-11-20 max cost | --max-cost-usd 20 | USD | [1] | 2026-11-20 passing exit code | 0 | | [1] | 2026-11-20 Sources: [1] Anthropic. "Test plugins with evals". https://code.claude.com/docs/en/plugin-evals.md Role: primary. Accessed: 2026-11-20T12:22:40Z Quote: "--threshold 0.8" (Supports: threshold flag) Quote: "--json results.json" (Supports: json flag) Quote: "--trust-plugin" (Supports: trust flag) Quote: "--model claude-sonnet-5" (Supports: model) Quote: "--judge-model claude-haiku-4-5" (Supports: judge model) Quote: "--max-cost-usd 20" (Supports: max cost) Quote: "| 0 | The suite passed: every case scored at or above `--threshold` and nothing else failed the run |" (Supports: passing exit code) Published: 2026-11-20T03:14:09Z Snapshot SHA-256: f612b7a35aa589f2beeeded25d8143492fde96a41688bf016856679a58211a98 (captured 2026-11-19T12:18:12Z) Verification: Method: Automated extraction from [1] with deepseek-flash (JSON output, thinking disabled). Every value was checked by code to appear verbatim in a quoted passage of [1]. On this check the text of [1] was unchanged since 2026-11-19T12:18:10Z, so the values verified then were kept. Cross-check: not-available at 2026-11-20T12:22:40Z. No independent second source is configured for this entry Last successful check: 2026-11-20T12:22:40Z Last attempt: 2026-11-20T12:22:40Z Notes and limitations: Extracted automatically from a single official source (section "Run evals in CI"). Related: https://latestn.com/tool/claude-code/agents-and-automation/run-prompt-fixed-interval How do I run a prompt repeatedly on a fixed interval in Claude Code? https://latestn.com/tool/claude-code/agents-and-automation/scale-concurrency-hosting How do I scale and handle concurrency when hosting the Claude Code Agent SDK? https://latestn.com/tool/claude-code/agents-and-automation/schedule-options What schedule options (frequency and timing) can I set for a scheduled task in Claude Code Desktop? https://latestn.com/tool/claude-code/agents-and-automation/rewind-files-to-previous-state How do I rewind files to a previous state in Claude Code using rewindFiles()? https://latestn.com/tool/claude-code/agents-and-automation/run-bundled-workflow How do I run a bundled workflow in Claude Code? https://latestn.com/tool/claude-code/agents-and-automation/run-non-interactively How do I run Claude Code non-interactively from the command line? https://latestn.com/tool/ All entries in AI tools Change log: Time | Event | Change | Source 2026-11-18T12:18:01Z | changed | max cost: — → --max-cost-usd 20 | [1] 2026-11-18T12:18:01Z | changed | json flag: — → --json results.json | [1] 2026-11-18T12:18:01Z | changed | cost ceiling: --max-cost-usd 20 → — | [1] 2026-11-18T12:18:01Z | changed | json output flag: --json results.json → — | [1] 2026-11-18T12:18:01Z | changed | failing exit code: 1 → — | [1] 2026-11-13T03:04:53Z | added | Entry created | — Changes for this entry as a feed: https://latestn.com/tool/claude-code/agents-and-automation/run-plugin-evals-in-ci/latest Challenge this entry: Open challenges: None How: If a value here is wrong or out of date, send what is wrong and a link to evidence. Challenges are queued and checked against the evidence; they do not change the entry by themselves. If the entry changes as a result, the change log says so. Form: https://latestn.com/dispute/tool/claude-code/agents-and-automation/run-plugin-evals-in-ci POST: https://latestn.com/dispute/tool/claude-code/agents-and-automation/run-plugin-evals-in-ci with claim, evidence_url, and optionally field and contact (form-encoded or JSON) GET: https://latestn.com/dispute/tool/claude-code/agents-and-automation/run-plugin-evals-in-ci?claim={what is wrong}&evidence_url={link} How to cite: LatestN. (2026). How do I run Claude Code plugin evals in CI and gate on the score? Revision 2, last verified 2026-11-20T12:22:40Z. https://latestn.com/tool/claude-code/agents-and-automation/run-plugin-evals-in-ci (accessed 2026-11-21T10:06:12Z). @misc{LatestN_tool_claude_code_agents_and_automation_run_plugin_evals_in_ci_r2, title = {How do I run Claude Code plugin evals in CI and gate on the score?}, author = {{LatestN}}, year = {2026}, howpublished = {\url{https://latestn.com/tool/claude-code/agents-and-automation/run-plugin-evals-in-ci}}, note = {Revision 2; digest sha256:4a582c1484325a42; last verified 2026-11-20T12:22:40Z; accessed 2026-11-21T10:06:12Z} } Reuse: License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) Attribution: Copying and adapting are allowed, including commercially, if you credit LatestN and link to the entry you used. Each entry ends with a ready-made citation. URL: https://latestn.com/tool/claude-code/agents-and-automation/run-plugin-evals-in-ci