hamelsmu

How to Correct an LLM Judge's Pass Rate With 100 Human Labels > Engineer's view. This is a health check that returns 200 while the database is down. Your uptime number then mixes real uptime with the check's blind spot. A model grading your outputs does the same thing. It passes some bad ones, so the score runs high and every real fix looks smaller than it was. > > TL;DR. The pass rate a grading model reports is part your product and part the grader's own mistakes. Measure those mistakes on a small set a person has checked, and you can subtract them back out. Explain Like I'm 5 You run a fruit stand, and a helper sorts apples into a good basket and a bad basket. She is quick, but she is kind. Most bruised apples land in the good basket too. So you count the good basket and feel proud. Nine apples in ten look fine. That number is partly your apples and partly her kindness. To find the real number, you test her. Hand her fifty apples you know are good and fifty you know are bruised. Count how many of each she gets right. Now you know how many bruised apples she lets through on a normal day. Take that many out of the good basket, and what is left is the true count. The Problem You have shipped this bug already, in a system with no model in it. Your uptime dashboard said 99.9% for a month. Users kept filing tickets about errors. After a day of digging you found the health check. It pinged an endpoint that returned 200 as long as the web process was alive. When the database was down, the check still passed. Nothing was wrong with the arithmetic. The dashboard faithfully counted what the check said. But the check had a blind spot, so the number mixed two things: real uptime and the check's habit of passing broken states. You could not fix the dashboard until you knew how often the check was wrong. An eval that uses an LLM judge has that bug at full size. An LLM judge is a model you prompt to grade each output of your feature as Pass or Fail. Its pass rate goes on the dashboard as if it were your product's pass rate. It is not. Judges make mistakes in both directions, and they lean lenient. One 2025 study of judges grading feedback on student code found they passed 96% of good answers but failed fewer than a quarter of bad ones. That leniency does two kinds of damage. The score runs high. Worse, it hides progress. When the judge passes most bad outputs anyway, fixing some of them barely moves the number. The fix: label about a hundred outputs by hand, measure how often the judge agrees on the good ones and on the bad ones, and use those two rates to undo the blend. The Fix: Measure the Judge's Two Error Rates, Then Undo Them !Where the judge's pass rate comes from Where does the wrong pass rate come from? From two numbers the dashboard never shows. The TPR is the share of truly good outputs the judge passes. The TNR is the share of truly bad outputs the judge fails. Here, Pass counts as the positive answer. The judge's pass rate is the good outputs it passes plus the bad outputs it lets through. A strict judge with a TPR of 0.92 and a TNR of 0.88 turns a true 70% into 68%. That is close, but only by luck: its two mistakes happen to cancel. The Rogan-Gladen correction runs that arithmetic backwards. It comes from medical testing, where a lab test has the same two error rates. Add the TNR to the judge's pass rate, subtract one, and divide by the TPR plus the TNR minus one. Why does a real fix look smaller? Because only part of any real change gets through. That part is the TPR plus the TNR minus one. For the strict judge it is 0.80, so a real eight-point gain shows as 6.4 points. !A real 8-point gain, as each judge reports it A lenient judge with a TNR of 0.24 keeps only a fifth of the change. Your team ships a real improvement, and the chart moves by 1.6 points. Can the correction rescue the lenient judge? No, and this is the part that bites. The correction divides by that same 0.20. Dividing by a small number turns small mistakes in your labels into huge swings. With a hundred labels, the lenient judge's corrected rate could be anything from 32% to 100%. Fix the judge first. What This Means for You When this matters. Any time a model grades your outputs and you report the share it passed. That covers a nightly eval, a release check, and a quality chart on a dashboard. It matters most when you compare two numbers: this release against the last one, or one prompt against another. How it affects you. The level is off, usually too high. The comparisons are shrunk, so real improvements look small and a prompt change that helped can look like noise. And it gets worse quietly. Swap in a stricter judge model, and your pass rate drops on a day you shipped nothing. What to do about it. Start with ten minutes and no new tooling. Find ten outputs you know are bad. A thumbs-down from a user, a bug report, or a few you break on purpose all work. Run your judge on them and count how many it passes. If it passes four, your judge lets roughly 40% of failures through. Every bad output it misses is counted on your dashboard as a success. Then do it properly. Label about a hundred outputs drawn from real traffic, half good and half bad. Measure the judge's TPR and TNR on them. Report the corrected rate with an interval next to the raw one. If the TPR plus the TNR comes out under about 0.3, stop. Do not correct. Rewrite the judge prompt or narrow it to one failure mode, and measure again. The next section shows the three pieces of code, and the check that tells you they work. Implementing It The change. Three people touch this. Each writes a little code, and none of it is hard. Getting the convention wrong is the usual failure, so name it once and use it everywhere. Role 1, whoever owns the labels. Keep the human verdicts beside the judge verdicts, one line per output, sampled from production. Balance the classes on purpose. Real traffic might be 90% good, and ten bad examples cannot measure a TNR. {"id": "conv-8812", "human": 1, "judge": 1, "split": "dev"} {"id": "conv-9034", "human": 0, "judge": 1, "split": "test"} {"id": "conv-9107", "human": 0, "judge": 0, "split": "test"} Use the dev rows while you rewrite the judge prompt. Measure the final rates on the test rows only, once. Rates measured on the rows you tuned against will look better than they are. Role 2, whoever owns the eval script. Compute the two rates, correct the pass rate, and refuse when the judge is too weak. def judge_rates(human, judge): # 1 = Pass, for both lists good = [j for h, j in zip(human, judge) if h == 1] bad = [j for h, j in zip(human, judge) if h == 0] return sum(good) / len(good), 1 - sum(bad) / len(bad) def corrected_pass_rate(p_obs, tpr, tnr, min_youden=0.30): youden = tpr + tnr - 1 if youden < min_youden: return None # report "judge too weak", not a number return min(1.0, max(0.0, (p_obs + tnr - 1) / youden)) The interval comes from resampling. Redraw the labeled rows and the production verdicts with replacement, recompute both rates and the corrected rate, and repeat 2,000 times. The middle 95% of the results is the interval. To compare two releases, draw one pair of rates per round and use it for both. The judge's error is shared, so part of it cancels. The full version is in code_example.py. If you would rather not write it, the MIT-licensed judgy package does the same thing: from judgy import estimate_success_rate theta, low, high = estimate_success_rate( test_labels=human, test_preds=judge, unlabeled_preds=production_verdicts) Role 3, whoever owns the dashboard and the review. Show the raw rate, the corrected rate, and its interval. Store the judge's TPR, TNR and exact model version next to each run. A new judge prompt or model needs new rates before its numbers can be compared. How you know it worked. Run two checks. First, use the dev rows to measure the rates, then correct the judge's pass rate on the test rows. The corrected number should land near the human pass rate on those rows. The raw number usually will not. Second, swap the convention on purpose. In the Code tab, a corrected 70.0% becomes 75.0% when TPR and TNR trade places. Some guides call a caught failure the positive case, so their TPR is your TNR. If two people's numbers disagree by a few points, check this first. When the Correction Is the Wrong Tool When the judge is barely better than a coin. The correction divides by the judge's two rates added together, minus one. Near zero, the interval covers almost everything. Improve the judge until both rates clear 0.8, then correct. When today's failures are a new kind. The correction assumes the judge misses failures as often in production as it did on your labels. In the Code tab, the labels say the judge fails 88% of bad outputs. Today's failures are subtler, and it fails only 60% of them. The true rate is 60%, and the stale correction reports 74%. Re-label a fresh sample when the product changes. When you need a verdict on one output. The correction fixes a rate across many outputs. It says nothing about whether a given reply was good, so it cannot gate a single response. And when you can afford to label everything. At fifty outputs a week, a person is the eval. Three questions before you adopt it: - Do both of the judge's rates clear 0.8 on outputs it never saw while you tuned it? - Were your labels drawn from the same traffic the judge is grading now? - Will the judge's prompt or model change before the next report? If so, its rates expire. Glossary - LLM judge — a model prompted to grade each output of your feature as Pass or Fail. - TPR — the share of truly good outputs the judge passes. - TNR — the share of truly bad outputs the judge fails. - Rogan-Gladen correction — arithmetic that turns a judge's pass rate back into the true one, using its two rates. - interval — the range the true pass rate probably sits in, after counting the noise in your labels. Further reading - How To Use Decision Models for Evals — Hamel Husain - validate-evaluator — Hamel Husain, evals-skills on GitHub - judgy — ai-evals-course on GitHub (MIT) - AI Evals: Everything You Need to Know — Hamel Husain --- Originally published at theaicommit.com, a daily AI engineering lab for software engineers. The original has an interactive version you can play with, and the full runnable code is on GitHub.