Neural and perceptual responses to a visual stimulus can be suppressed by the addition of both spatially overlapping and spatially adjacent contextual stimuli. We investigated the temporal characteristics of these suppressive interactions in psychophysical contrast masking experiments using Gabor and grating stimuli with a spatial frequency of 4 cycles per degree. We found that the time course of masking strongly depended on mask orientation. Most interestingly, masking by a spatially overlaid, iso-oriented mask was strongest when the target was presented immediately before or immediately after the mask. This masking was transient, presumably caused by the neural responses to mask onset and offset. Adding a surround to the mask modulated the backward masking effect, but only when the target and the central mask were iso-oriented. Our results provide evidence for a surround suppression mechanism that affected the transient responses to the mask onset, but not the responses to the mask offset. Together, these results demonstrate how the effects of spatial context in visual processing critically depend on stimulus timing.