9 ways to evaluate an LLM

Lesson 2.6 · 9 min read · by Vinay C

We'll cover the following

Visual summary

Infographic summarizing 9 ways to evaluate an LLM: You built an LLM app and it gives answers. How do you know they are good? There are nine common ways to measure it.…

Download imageFree to share, please link back to this lesson.

You built an LLM app and it gives answers. How do you know they are good? There are nine common ways to measure it. Some count words, some use another LLM as a grader, some use people, and one is built just for agents.

The running example

  • Question: "Describe the cat."
  • Reference (correct) answer: "The cat sat on the mat."
  • Model output: "A cat was sitting on the mat."

The nine at a glance

MethodWho gradesQuestion it answersBest for
1. BLEUWord match (precision)Is what the model said correct?Translation
2. ROUGEWord match (recall)Did the model cover everything?Summarization
3. BERTScoreEmbedding similarityDoes it mean the same thing?Anything where paraphrasing is fine
4. LLM-as-a-JudgeOne LLM + rubricWhich answer is better, or what score?Open-ended text
5. G-EvalOne LLM, chain of thoughtDoes it meet this criterion?Open-ended text, finer scores
6. LLM JurySeveral LLMs, averagedWhat do independent judges agree on?Reliable scores when you can pay more
7. DAGLLM at each node of a flowchartWhich checks pass, in order?Strict, repeatable, auditable rules
8. Human evaluationPeopleDo humans agree it is good?The most trusted result; calibrating the rest
9. Trajectory accuracyCompare steps to expected stepsDid the agent take the right path?Agents

Group 1: compare the output to a correct answer

No AI judge needed. These compare the output to a reference answer you already have.

BLEU: "Is what the model said correct?" It checks every word chunk (n-gram) in the output against the reference. Score = matched chunks / total chunks in the output. That is precision: it punishes saying things that are not in the reference. "The cat." scores high on precision but says almost nothing (BLEU adds a brevity penalty to partly fix this). A long answer full of extra words scores low.

ROUGE: "Did the model cover everything?" It flips the direction: every chunk in the reference is checked against the output. That is recall: it punishes leaving things out. "The cat." matches only 2 of 6 reference words, so ROUGE is low. The long answer contains every reference word, so ROUGE is high even with the noise.

BLEU vs ROUGE in one line: BLEU asks "is everything I said correct?" ROUGE asks "did I say everything I should have?"

BERTScore: "Does it mean the same thing?" BLEU and ROUGE only count exact word matches, so "sat" vs "sitting" is a miss. BERTScore turns each word into a vector that captures meaning and matches words by how close their vectors are. "sat" and "sitting" are close, so the score is high.

Group 2: another LLM grades the answer

These four methods are all LLM-as-a-Judge: a strong model is the grader. They differ in how the judge is set up. For how this goes wrong and how to calibrate it, see "Can you trust an LLM judge?".

  • LLM-as-a-Judge (the base idea): give a judge LLM a rubric and the answers. Two modes: pairwise (A vs B, pick the better) or single score (grade one answer 1 to 5). Watch for a preference for longer answers, for whichever comes first, and for its own writing style.
  • G-Eval (a specific recipe): you give only the criteria, such as "Is the summary factually consistent?". The judge first writes its own step-by-step evaluation plan (chain of thought), then follows it. Scores are smoother because it uses the model's token probabilities: 60% sure it is a 4 and 40% sure it is a 3 gives 3.6, not a flat 4. This lines up better with human ratings than a plain "rate 1 to 5".
  • LLM Jury (many judges instead of one): run several judge models in parallel (for example GPT, Claude and Gemini), score independently, then average. One judge has biases; a panel cancels many out. It costs more but is more reliable.
  • DAG (a flowchart of yes/no checks): "acyclic" means the flow only moves forward. Instead of "rate this overall", break grading into narrow questions, each answered by an LLM. The answer decides the next step, and different paths end in different scores.

DAG example: grading a customer support reply

  1. Did the reply answer the customer's question? No: score 0, stop. Yes: next check.
  2. Did it invent a policy that does not exist? Yes: score 3, stop. No: next check.
  3. Is the tone polite? No: score 7. Yes: score 10.

Dealbreakers are checked first, the logic is fixed (same input, same path) and you can see exactly why a score was given. G-Eval is flexible but fuzzy. DAG is strict and auditable.

Group 3: people

Human evaluation: people read the outputs and score them against clear criteria such as accuracy, helpfulness and tone. Have several people score the same outputs and check that they agree with each other. It is the most trusted method but slow and expensive, so use it to calibrate the automated methods.

Group 4: AI agents

Trajectory accuracy: "did the agent take the right steps?" Agents take actions in sequence. A correct final answer can hide a bad process, so this scores the path, not just the result.

Task: "Book the cheapest flight from Delhi to Mumbai tomorrow and email me the confirmation."

Expected                              Actual
1. search_flights (Delhi->Mumbai)       1. search_flights (Delhi->Mumbai)
2. sort_by_price                        (skipped)
3. book_flight (cheapest)               2. book_flight (first result)
4. send_email (confirmation)            3. send_email (confirmation)

The final message says "Booked! Email sent." and looks perfect, but the agent never compared prices, so it may have booked the wrong flight. Trajectory score: 3 of 4 expected steps matched = 0.75. Match strictly (exact order), in order (extra steps allowed) or in any order, depending on how rigid the task is.

Which one should you use?

SituationUse
Have a reference answer and need speedBLEU, ROUGE, BERTScore
Open-ended text, no referenceG-Eval or LLM-as-a-Judge
Need reliable scores and can pay moreLLM Jury
Need strict, repeatable, explainable rulesDAG
Need the most trusted resultHuman evaluation
Evaluating agentsTrajectory accuracy

In practice, teams combine several: cheap automated metrics on everything, LLM judges at scale, and humans on a sample to keep the judges honest.

Check yourselfYou are summarizing documents and the main risk is missing a key point. BLEU or ROUGE?

ROUGE. It is recall-based: it checks whether everything in the reference shows up in the output. BLEU is precision-based and would not punish omissions.

Check yourselfThe model says "sitting" where the reference says "sat". Why does BLEU/ROUGE score it low and what fixes it?

They only count exact word matches, so a paraphrase is a miss. BERTScore compares meaning through embeddings, so "sat" and "sitting" score as close.

Check yourselfAn agent ends with "Booked! Email sent." Is that enough to call the run correct?

No. The final answer can hide a bad process (it skipped the price comparison). Use trajectory accuracy to score the steps against the expected path.