What is the best paradigm to recognize objects — discriminative inference (fast but potentially prone to shortcut learning) or using a generative model (slow but potentially more robust)?

This paper builds on recent advances in generative modeling that turn text-to-image models into classifiers, allowing the authors to study their behavior and compare them against discriminative models and human psychophysical data.

Four Intriguing Properties

The researchers report four emergent properties of generative classifiers:

1. Human-like shape bias — A record-breaking 99% shape bias for Imagen, meaning these models recognize objects primarily by their shape rather than texture, just like humans do.

2. Out-of-distribution accuracy — Near human-level accuracy on out-of-distribution data, suggesting generative classifiers generalize more robustly than their discriminative counterparts.

3. Error alignment with humans — State-of-the-art alignment with human classification errors. When generative classifiers get things wrong, they tend to make the same mistakes humans do.

4. Understanding perceptual illusions — These models are vulnerable to human-like illusions, such as the famous “Rabbit-Duck” image. This suggests they have learned perceptual representations that share fundamental properties with human vision.

Why It Matters

While the current dominant paradigm for modeling human object recognition is discriminative inference, zero-shot generative models approximate human object recognition data surprisingly well. This challenges the long-held assumption that discriminative models are the best path to human-like perception and opens up interesting possibilities for using generative models as a lens into understanding human cognition.

Read the paper