Write the Review Rubric Before the AI Judge Scores the Work

An AI judge is not a control until the rubric is written.

Teams are starting to use LLMs to grade AI output, review support drafts, score research summaries, triage tickets, judge coding tasks, and decide whether agent work is good enough to move forward.

That can be useful.

It can also turn into a quiet approval gate that nobody validated.

The problem is not that LLM judges are always bad. The problem is that they are persuasive. They produce a score, a rationale, and a clean summary. That makes the workflow feel governed even when the judge is applying hidden preferences.

Recent research keeps pointing at the same failure pattern: model judges can be consistent and still wrong in predictable ways.

Norman, Rivera, and Hughes evaluated LLM judges across agreement, consistency, and bias. Their paper argues that exact-match agreement can overstate judge quality because it does not correct for chance. They also found that high test-retest reliability can coexist with severe position bias. A judge can give the same answer repeatedly and still prefer the wrong thing for the wrong reason.

BabelJudge makes the same point from another angle. It tests position bias, verbosity bias, order inconsistency, cross-lingual degradation, and agentic trajectory failures. The agentic part matters for real workflows. A reviewer agent may need to notice wrong tool arguments, swapped tools, hallucinated calls, or missing steps, not just pick the smoother paragraph.

JudgeBiasBench is useful because it names the core risk plainly: judges can be influenced by task-irrelevant attributes like style, verbosity, formatting, or context cues. That is exactly what happens in knowledge work. The polished draft looks better than the correct but awkward one. The longer research summary feels more complete. The markdown table feels more rigorous.

This is why the review rubric has to come first.

Before a team lets an AI judge score workflow output, the team should write down seven things:

1. What task is being judged.
2. What evidence the judge is allowed to use.
3. What must be true for approval.
4. What forces rejection even if the output reads well.
5. Which bias probes the judge must pass.
6. What happens when judges or humans disagree.
7. When the workflow stops and routes to a human.

The bias probes are the part most teams skip.

Give the judge a longer answer that is worse.

Swap answer order and see whether the verdict changes.

Give it a cleanly formatted answer with missing evidence.

Give it a tool trace where the final summary sounds right but the tool arguments are wrong.

Give it the same task in the language, domain, and format the workflow actually uses.

If the judge cannot pass those checks, it should not be an approval gate. It can still help with triage, summarization, first-pass review, or pointing a human reviewer toward likely issues. That is a different level of authority.

The stronger claim is this: judge prompts are workflow controls.

They are not just evaluation plumbing. They decide what counts as good work, what gets escalated, what gets ignored, and what gets shipped.

So the safe sequence is not:

1. Add AI judge.
2. See what happens.
3. Tune when people complain.

The safe sequence is:

1. Write the rubric.
2. Build known-good and known-bad examples.
3. Run bias probes.
4. Define the human escalation rule.
5. Only then give the judge workflow authority.

Reasoning models do not remove this step. Wang et al. found that large reasoning models still show judging biases, including bandwagon, authority, position, distraction, and superficial reflection effects. A model saying it thought carefully is not the same thing as a validated review process.

Soumik's bias-mitigation study also pushes against a lazy answer: "just use the best model." The paper found that debiasing strategy and model setup can change judge performance materially. Bigger is not the same as validated.

For agentic AI security work, the recommendation is simple.

Treat every AI judge like a workflow gate. If it can approve, reject, route, close, score, or escalate work, it needs the same discipline as any other gate: criteria, evidence, failure cases, sampling, escalation, and review.

The rubric is not paperwork.

It is the control surface.