As image-generation models become a standard part of consumer applications, editing photographs has emerged as a common use case. Despite steady gains in fidelity, these edits tend to introduce subtle but unrequested alterations, and this effect compounds when outputs are repeatedly fed back as inputs. On human faces, this accumulation gradually erodes the subject’s identity, a phenomenon we refer to as identity drift. Because people are acutely sensitive to faces, they sense that something is off even when they cannot say what changed. This benchmark measures identity drift on 199 photographs of real people. Each photograph gets a prompt chain of five everyday edit prompts on clothing, objects or text in the scene, none of which concerns the face. Every model applies the prompts in turn, each to its own previous output, and every step is checked automatically for whether the requested edit was made. Human annotators then compare the faces after the fifth edit in pairwise matches, with the original face shown as reference, and judge which looks more similar to it. The 322,417 votes cast across 60,800 such matches are aggregated into ratings with a Bradley–Terry model, such that a higher rating indicates less identity drift; the methodology details matchmaking and score estimation.
Figure A2. Each model's standing within each prompt chain category, scored on overall preference. Read a column: who is best in that category. Darkest is first. Hover or tap a cell to read its score, 95% interval and sample size; click a category to sort the rows by it. The number beside each model is its overall rank, so a jump reads as a jump. A cell reports a rank only above 500 matches for that model and is hatched below it, a rank on a handful of matchups is noise, not a result. How many prompt chains a category rests on is in its header and every tooltip: a narrow category asks a narrower question.
We use crowd intelligence to deliver real, diverse human feedback at scale for model evaluation and post-training.
