Answer: Judges are set up by calling judge methods such as t.judge.factuality(expected).atLeast(0.8), t.judge.summarizes(expected).atLeast(0.8), t.judge.closedQA("names the root cause").gate(1), and t.judge.sql(expected).atLeast(0.9); each judge grades t.reply unless you pass { on } with another value. Judge options can be set in the project config, the eval, the case, or the call, and the nearest value wins; the text does not describe setting up a judge host. Status: Confirmed Entry: tool/cursor/agents-and-automation/eval-judges Revision: 1 Digest: sha256:3ff08347348972e1 (changes whenever the content changes) Timeline: Created: 2026-11-13T03:08:50Z Last changed: 2026-11-13T03:08:50Z Last verified: 2026-11-20T12:33:44Z Next check: 2026-11-21T12:33:44Z Check interval: Daily Expected change: No fixed schedule Data: Field | Value | Source | As of factuality judge threshold | 0.8 | [1] | 2026-11-20 summarizes judge threshold | 0.8 | [1] | 2026-11-20 closedQA judge gate | 1 | [1] | 2026-11-20 sql judge threshold | 0.9 | [1] | 2026-11-20 default graded value | t.reply | [1] | 2026-11-20 judge option precedence | the nearest value wins | [1] | 2026-11-20 Sources: [1] Anysphere. "cursor.com/docs/agent-sdk/reference/evals.md". https://cursor.com/docs/agent-sdk/reference/evals.md Role: primary. Accessed: 2026-11-20T12:33:44Z Quote: "t.judge.factuality(expected).atLeast(0.8);" (Supports: factuality judge threshold) Quote: "t.judge.summarizes(expected).atLeast(0.8);" (Supports: summarizes judge threshold) Quote: "t.judge.closedQA("names the root cause").gate(1);" (Supports: closedQA judge gate) Quote: "t.judge.sql(expected).atLeast(0.9);" (Supports: sql judge threshold) Quote: "Each judge grades `t.reply` unless you pass `{ on }` with another value." (Supports: default graded value) Quote: "Set judge options in the project config, the eval, the case, or the call, and the nearest value wins." (Supports: judge option precedence) Snapshot SHA-256: 30875c6a9ce5f98e680c15dd555ef5286ce3c5b5a73928c865de2b230687bfbe (captured 2026-11-13T03:08:50Z) Verification: Method: Automated extraction from [1] with deepseek-flash (JSON output, thinking disabled). Every value was checked by code to appear verbatim in a quoted passage of [1]. On this check the text of [1] was unchanged since 2026-11-13T03:08:50Z, so the values verified then were kept. Cross-check: not-available at 2026-11-20T12:33:44Z. No independent second source is configured for this entry Last successful check: 2026-11-20T12:33:44Z Last attempt: 2026-11-20T12:33:44Z Notes and limitations: Extracted automatically from a single official source (section "Judges"). Related: https://latestn.com/tool/cursor/agents-and-automation/evals-folder-location Where do Cursor Agent SDK evals live in a project? https://latestn.com/tool/cursor/agents-and-automation/explore-subagent-usage How do I make Cursor Agent use the Explore subagent for broad searches? https://latestn.com/tool/cursor/agents-and-automation/expose-tag-artifact-tool How do I expose tag_artifact to the model in Cursor Agent SDK? https://latestn.com/tool/cursor/agents-and-automation/enable-computer-use How do I enable computer use for a Cursor self-hosted worker? https://latestn.com/tool/cursor/agents-and-automation/enable-pr-routing-approval How do I enable PR Routing & Approval in Cursor? https://latestn.com/tool/cursor/agents-and-automation/eval-configuration How do I configure Cursor Evals with an evals.config.ts file? https://latestn.com/tool/ All entries in AI tools Change log: Time | Event | Change | Source 2026-11-13T03:08:50Z | added | Entry created | — Changes for this entry as a feed: https://latestn.com/tool/cursor/agents-and-automation/eval-judges/latest Challenge this entry: Open challenges: None How: If a value here is wrong or out of date, send what is wrong and a link to evidence. Challenges are queued and checked against the evidence; they do not change the entry by themselves. If the entry changes as a result, the change log says so. Form: https://latestn.com/dispute/tool/cursor/agents-and-automation/eval-judges POST: https://latestn.com/dispute/tool/cursor/agents-and-automation/eval-judges with claim, evidence_url, and optionally field and contact (form-encoded or JSON) GET: https://latestn.com/dispute/tool/cursor/agents-and-automation/eval-judges?claim={what is wrong}&evidence_url={link} How to cite: LatestN. (2026). How do I set up judges and a judge host for Cursor Evals? Revision 1, last verified 2026-11-20T12:33:44Z. https://latestn.com/tool/cursor/agents-and-automation/eval-judges (accessed 2026-11-21T10:03:15Z). @misc{LatestN_tool_cursor_agents_and_automation_eval_judges_r1, title = {How do I set up judges and a judge host for Cursor Evals?}, author = {{LatestN}}, year = {2026}, howpublished = {\url{https://latestn.com/tool/cursor/agents-and-automation/eval-judges}}, note = {Revision 1; digest sha256:3ff08347348972e1; last verified 2026-11-20T12:33:44Z; accessed 2026-11-21T10:03:15Z} } Reuse: License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) Attribution: Copying and adapting are allowed, including commercially, if you credit LatestN and link to the entry you used. Each entry ends with a ready-made citation. URL: https://latestn.com/tool/cursor/agents-and-automation/eval-judges