Same article. Different batch. Different answer?
An aggregate score can stay close while individual decisions change. Compare the same article under two serving policies.
Example sets use the fixed-padding comparison. Changing the right-hand policy keeps your article selected.
1. Keep the article fixed
Full summary shown below. Model input is capped at 256 tokens.
Compare predictions and scores
Read the experiment boundaries
The example filter always compares batch 16 with batch one at fixed padding. It stays fixed when you switch policy, so the selected article does not disappear.
Each example comes from the 1,156 validation articles excluded from quantization diagnosis. The original checkpoint was selected using full validation. The INT8 model keeps the first FFN down projection in FP32 and quantizes the other 168 Linear modules.
Batch 16 versus batch one at fixed padding changed 67 INT8 decisions; switching from batch 16 to batch one at actual length changed 73. FP32 decisions stayed identical. These are disagreements, not counts of additional errors. The two comparisons can involve different articles.
The dataset labels are mapped newsletter sections, not independently annotated semantic topics. Articles are ordered by the frozen validation order, not hand-picked for a dramatic example. A changed prediction alone does not establish why the numerical output changed.
These controls replay recorded results. They do not run inference or isolate batch companions from batch shape and kernel behavior.