In studies of visual memory, an image identical to that previously seen is invariably recognized more accurately than any image that is different. This identity superiority provides the empirical foundation for the image-based theory of object and scene recognition. Here we present evidence to the contrary. In an “old–new” recognition task, a face less-occluded than its “old” counterpart was recognized more accurately than the “old” face itself. The same effect was also found with natural scenes. The superiority of a less-occluded image weakened when occlusion was further removed, indicating the visual system's ability to recover from occlusion is limited. When the images were inverted, the effect disappeared. Our findings support the classic notion that the visual system performs active abstraction and organization on sensory information in order to encode it into a memory representation.