Stereopsis, the ability to sense the world in three dimensions (3D) from pairs of retinal images, functions when both images have corresponding elements. When observers view stereograms lacking a global match, they do not perceive 3D structure, whereas several cortical areas encode stereoscopic depth in the disparity energy. Whether these neural representations are exploited or ignored in perceptual decisions remains elusive. By combining contrast-reversal and delay between stereo images, we found that disparity-energy signals mediate the reversal of stereoscopic depth judgments. A crisp, adjacent plane of reference was crucial for the signal to be used in the judgments. Disparity discrimination relies on the disparity-energy signal when the stimulus has no global binocular match and is accompanied by a fixed surface of reference.