Evaluation¶

eval.py loads every stage's checkpoint that exists on disk and puts them side by side on the same prompts — so you can see what pretraining, SFT, LoRA, and DPO each changed. It needs no training state, so it runs standalone any time after the stages you want to compare have finished. Missing checkpoints (LoRA is optional) are skipped with a note.

Summary

Script: eval.py · Loads: ckpt_base.pt, ckpt_sft.pt, ckpt_lora.pt, ckpt_dpo.pt (whichever exist) · Run: python eval.py

The four probes¶

1. Instruction following on a hard prompt¶

A prompt with unusual, checkable words (spoon, bird, brave). For each stage the script reports the required-words coverage, sentiment, and whether the model stopped on its own (emitted <|endoftext|> before the token cap). Pretraining typically ignores the instruction; SFT and LoRA follow it and stop.

2. A sad instruction¶

"Summary: A dog loses his toy and cries." This is where the DPO model shows its bias: SFT obeys the sad summary, while a DPO model trained with the sentiment reward drags the story back toward a happier tone — visible in the sentiment score.

3. Mean reward over a fixed pool¶

One number per stage: the mean programmatic reward over 64 fixed eval prompts (the same pool and seed as dpo.py). This is the single trend line that should climb from pretrain → sft → dpo when DPO is working.

============== mean reward (64 prompts) ==============
pretrain  +0.0xx
sft       +0.1xx
dpo       +0.2xx

4. Next-word probability probe¶

No sampling noise — a direct look at the distribution. After "The dog felt very", the script prints P(happy), P(sad), P(scared), P(excited) for each stage, so you can watch the preference-tuned model shift probability mass toward positive continuations.

python eval.py

Reproducibility

The eval pool is drawn with the same seed (np.random.default_rng(0)) as the DPO stage, so the reward numbers here line up with what you saw during training.

📄 Source: eval.py