As image-generation models become a standard part of consumer applications, editing photographs has emerged as a common use case. Despite steady gains in fidelity, these edits tend to introduce subtle but unrequested alterations, and this effect compounds when outputs are repeatedly fed back as inputs. On human faces, this accumulation gradually erodes the subject’s identity, a phenomenon we refer to as identity drift. Because people are acutely sensitive to faces, they sense that something is off even when they cannot say what changed. This benchmark measures identity drift on 199 photographs of real people. Each photograph gets a prompt chain of five everyday edit prompts on clothing, objects or text in the scene, none of which concerns the face. Every model applies the prompts in turn, each to its own previous output, and every step is checked automatically for whether the requested edit was made. Human annotators then compare the faces after the fifth edit in pairwise matches, with the original face shown as reference, and judge which looks more similar to it. The 322,417 votes cast across 60,800 such matches are aggregated into ratings with a Bradley–Terry model, such that a higher rating indicates less identity drift; the methodology details matchmaking and score estimation.
Table 1. Overall standings, Multi-step image editing benchmark. Scores from a Bradley–Terry fit rescaled to Elo; intervals are 95% Wald intervals. Click any row to view sample outputs in Figure 1.
Figure 2. Overall Elo against the provider's list price, cost on a log scale. Teal-ringed models are the Pareto frontier, no model is both cheaper and better, joined by a step line, so picking a budget on the x-axis reads off the best model at or under it. Faded models are dominated; whiskers are the score 95% CI. Prices are hand-maintained snapshots of provider list prices. Hover a model for exact numbers.
We use crowd intelligence to deliver real, diverse human feedback at scale for model evaluation and post-training.
