Visual texture segregation is believed to be performed preattentively. Recent evidence, however, suggests that attention does play an important role. Using visual evoked potentials (VEPs), we investigated the effect of different tasks on texture segregation. Stimuli consisted of Gabor-filtered binary noise patterns. In segregated stimuli, local texture orientation contrasts defined global checkerboard patterns. VEP responses specific to texture segregation were obtained by computing the difference between VEPs to homogeneous and segregated stimuli. Four conditions were examined that required attending either the global pattern, the local structure, random numbers displayed on the screen, or a series of tones. Responses specific to texture segregation were dominated by two occipital negativities peaking around 110 and 230 ms. The earlier one was not affected by the task, whereas the later one was completely abolished when the subjects attended to either numbers or tones (p = .0005 and p = .006, respectively). The results suggest that early stages of texture segregation are not affected by attention, whereas task relevance is crucial for later processes. The timing is compatible with a recurrent processing pattern with initial bottom-up processing of basic stimulus characteristics and a subsequent top-down flow of higher level modulatory information. As attention effects occur across modalities, they cannot be simply explained by competition within the visual cortex.