UNIST's REFINED-BIAS replaces a shape-bias score that rose when vision models failed at texture

Early cue-conflict results set a training target. Models seemed to favor texture where humans favor shape, so the prevailing view was that models should be trained to put shape first ^[3]. Later studies could not agree on whether shape preference even correlated with performance ^[4]. The UNIST team traced that confusion to the benchmark itself. The old method counted only the ratio of a model's choices, not how well the model used each cue ^[5].

In the team's example, the strong model's correct shape answers outnumber the weak model's by a factor of ten ^[1], and both still score 80 percent ^[6]. According to the team, the method could not pick out a weak model whose shape figure rose only because it failed to recognize texture at all ^[6].

REFINED-BIAS isolates each cue before scoring it. Shape images have their surface patterns erased, texture images are cut and rearranged to hide the form, and any answer a model can produce now counts ^[10]. The 6,000 new images are spread over 20 categories, an average of 300 per category [c9, d2].

For anyone deciding where training effort goes, the findings move the target. If strong models are separated by the total visual information they draw on ^[11], then work aimed at raising a model's shape share was aimed at a number that could climb for the wrong reason ^[6]. The new scoring does keep the earlier result on transformer and text-image models ^[12]. UNIST's announcement does not list which published model comparisons would change.

Yoo Jae-jun, the professor at UNIST's Graduate School of Artificial Intelligence who led the team, said a benchmark is "a reference point that sets the direction of future development" [c1, c13]. "This research will play a central role in improving model architectures and training methods by precisely diagnosing how visual AI uses information," he said ^[14].

If other groups rerun their published comparisons on REFINED-BIAS and the rankings reshuffle, papers that trained models toward shape will need rereading. If the reruns reproduce the order cue-conflict gave, the distortion is real in principle and small in practice. That result would show the case against the old test is overstated. I'd expect any reshuffle to show up among weaker models, because those are the ones whose shape share the ratio could inflate ^[6].