Answer: There are six grader types: regex, tool_used, tool_order, file_exists, llm, and baseline. You choose and weight them via a grader's frontmatter, which sets its type and optionally a weight that makes it count for more of the run's score. Status: Confirmed Entry: tool/claude-code/agents-and-automation/choose-and-weight-graders Revision: 3 Digest: sha256:19bb3979586e6861 (changes whenever the content changes) Timeline: Created: 2026-11-13T03:04:54Z Last changed: 2026-11-19T12:18:11Z (judge model option: — → --judge-model sonnet) Last verified: 2026-11-20T12:22:42Z Next check: 2026-11-21T12:22:42Z Check interval: Daily Expected change: No fixed schedule Data: Field | Value | Source | As of grader types | regex | [1] | 2026-11-20 weight setting | weight | [1] | 2026-11-20 custom-code graders | There are no custom-code graders. | [1] | 2026-11-20 judge model option | --judge-model sonnet | [1] | 2026-11-20 Sources: [1] Anthropic. "Test plugins with evals". https://code.claude.com/docs/en/plugin-evals.md Role: primary. Accessed: 2026-11-20T12:22:42Z Quote: "Of the six types, `regex`, `tool_used`, `tool_order`, and `file_exists` are computed from the transcript and files and cost nothing, while `llm` and `baseline` call a judge model and add to the run's cost." (Supports: grader types) Quote: "A grader's frontmatter sets its `type`, and optionally a `weight` that makes it count for more of the run's score" (Supports: weight setting) Quote: "There are no custom-code graders." (Supports: custom-code graders) Quote: "Pass `--judge-model sonnet` or a full model ID to choose the judge yourself." (Supports: judge model option) Published: 2026-11-20T03:14:09Z Snapshot SHA-256: f612b7a35aa589f2beeeded25d8143492fde96a41688bf016856679a58211a98 (captured 2026-11-19T12:18:12Z) Verification: Method: Automated extraction from [1] with deepseek-flash (JSON output, thinking disabled). Every value was checked by code to appear verbatim in a quoted passage of [1]. On this check the text of [1] was unchanged since 2026-11-19T12:18:11Z, so the values verified then were kept. Cross-check: not-available at 2026-11-20T12:22:42Z. No independent second source is configured for this entry Last successful check: 2026-11-20T12:22:42Z Last attempt: 2026-11-20T12:22:42Z Notes and limitations: Extracted automatically from a single official source (section "Choose and weight graders"). Related: https://latestn.com/tool/claude-code/agents-and-automation/choose-display-mode How do I choose the display mode for my Claude Code agent team? https://latestn.com/tool/claude-code/agents-and-automation/choose-hook-type-sdk When should I use which hook type in the Claude Code SDK? https://latestn.com/tool/claude-code/agents-and-automation/claude-write-save-workflow How do I have Claude write and save a workflow in Claude Code? https://latestn.com/tool/claude-code/agents-and-automation/capture-session-id How do I capture the session ID in Claude Code SDK? https://latestn.com/tool/claude-code/agents-and-automation/change-system-prompt-existing-session How do I change the system prompt of an existing Claude Code session? https://latestn.com/tool/claude-code/agents-and-automation/check-running-agent-work How do I check on work that Claude Code agents are running in the background? https://latestn.com/tool/ All entries in AI tools Change log: Time | Event | Change | Source 2026-11-19T12:18:11Z | changed | judge model option: — → --judge-model sonnet | [1] 2026-11-19T12:18:11Z | changed | grader types: — → regex | [1] 2026-11-19T12:18:11Z | changed | judge model flag: `--judge-model sonnet` → — | [1] 2026-11-19T12:18:11Z | changed | weight setting: `weight` → weight | [1] 2026-11-19T12:18:11Z | changed | judge-model grader types: `llm` and `baseline` → — | [1] 2026-11-19T12:18:11Z | changed | free grader types: `regex`, `tool_used`, `tool_order`, and `file_exists` → — | [1] 2026-11-19T12:18:11Z | changed | number of grader types: six → — | [1] 2026-11-17T12:28:15Z | changed | judge model flag: — → `--judge-model sonnet` | [1] 2026-11-17T12:28:15Z | changed | arm setting: arm → — | [1] 2026-11-17T12:28:15Z | changed | weight setting: weight → `weight` | [1] 2026-11-17T12:28:15Z | changed | judge-model grader types: llm → `llm` and `baseline` | [1] 2026-11-17T12:28:15Z | changed | free grader types: regex → `regex`, `tool_used`, `tool_order`, and `file_exists` | [1] 2026-11-13T03:04:54Z | added | Entry created | — Changes for this entry as a feed: https://latestn.com/tool/claude-code/agents-and-automation/choose-and-weight-graders/latest Challenge this entry: Open challenges: None How: If a value here is wrong or out of date, send what is wrong and a link to evidence. Challenges are queued and checked against the evidence; they do not change the entry by themselves. If the entry changes as a result, the change log says so. Form: https://latestn.com/dispute/tool/claude-code/agents-and-automation/choose-and-weight-graders POST: https://latestn.com/dispute/tool/claude-code/agents-and-automation/choose-and-weight-graders with claim, evidence_url, and optionally field and contact (form-encoded or JSON) GET: https://latestn.com/dispute/tool/claude-code/agents-and-automation/choose-and-weight-graders?claim={what is wrong}&evidence_url={link} How to cite: LatestN. (2026). What grader types exist for Claude Code plugin evals and how do I choose and weight graders? Revision 3, last verified 2026-11-20T12:22:42Z. https://latestn.com/tool/claude-code/agents-and-automation/choose-and-weight-graders (accessed 2026-11-21T10:05:02Z). @misc{LatestN_tool_claude_code_agents_and_automation_choose_and_weight_graders_r3, title = {What grader types exist for Claude Code plugin evals and how do I choose and weight graders?}, author = {{LatestN}}, year = {2026}, howpublished = {\url{https://latestn.com/tool/claude-code/agents-and-automation/choose-and-weight-graders}}, note = {Revision 3; digest sha256:19bb3979586e6861; last verified 2026-11-20T12:22:42Z; accessed 2026-11-21T10:05:02Z} } Reuse: License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) Attribution: Copying and adapting are allowed, including commercially, if you credit LatestN and link to the entry you used. Each entry ends with a ready-made citation. URL: https://latestn.com/tool/claude-code/agents-and-automation/choose-and-weight-graders