As image-generation models become a standard part of consumer applications, editing photographs has emerged as a common use case. Despite steady gains in fidelity, these edits tend to introduce subtle but unrequested alterations, and this effect compounds when outputs are repeatedly fed back as inputs. On human faces, this accumulation gradually erodes the subject’s identity, a phenomenon we refer to as identity drift. Because people are acutely sensitive to faces, they sense that something is off even when they cannot say what changed. This benchmark measures identity drift on 199 photographs of real people. Each photograph gets a prompt chain of five everyday edit prompts on clothing, objects or text in the scene, none of which concerns the face. Every model applies the prompts in turn, each to its own previous output, and every step is checked automatically for whether the requested edit was made. Human annotators then compare the faces after the fifth edit in pairwise matches, with the original face shown as reference, and judge which looks more similar to it. The 322,417 votes cast across 60,800 such matches are aggregated into ratings with a Bradley–Terry model, such that a higher rating indicates less identity drift; the methodology details matchmaking and score estimation.
Full protocol in the global methodology.
We use crowd intelligence to deliver real, diverse human feedback at scale for model evaluation and post-training.
