Simultaneous contrast phenomena typically contain two patches of light that have identical luminance levels but have different perceived brightness levels (e.g., a patch with a black surround appears brighter than a patch with a white surround). Explanations of such phenomena generally assume that brightness relationships in the visual scene are determined by some form of early neural interaction (Hering-type explanation), inferences about illumination (Helmholtz-type explanation), or comparisons among multiple luminance frameworks (Gestalt-type explanation). We examine whether simultaneous contrast phenomena arise from a general process in which the visual system eliminates spatial frequencies lower than the fundamental frequency of the area of interest (an efficiency-type explanation). We examine a wide variety of contrast and assimilation illusions (standard SC, articulated surrounds, blurred-surround, Adelson's illusion snake and checker-shadow illusions, Purves and Lotto illusions, Logvinenko diamonds, Anderson and Winawer illusions, White's effect, the Bressan's Dungeon illusion, and a new class of illusions which we refer to as gradient-gradient illusions). When low-spatial frequencies are removed from these displays (i.e., when the displays are passed through a high-pass filter with a low spatial frequency cut-off), the physical values of the resulting images follow the same pattern as that of the perceived brightness. The results suggest that the test patches are actually physically different over the range of relevant spatial frequencies. We relate this approach to other spatial frequency models of brightness illusions (Blakeslee and McCourt, 1999; Dakin and Bex, 2003) and to a new model by Ioannides, Johnston, & Griffin (2006).