Separating objects from their backgrounds is a fundamental aspect of early visual processing and a pre-requisite for object recognition. Here we use multi-input nonlinear analysis methods and high-density EEG recordings to separately track the temporal evolution of figure and background activity through several visual areas identified by fMRI mapping studies. The figure and background regions of simple texture-defined forms were tagged by independent m-sequences. Binary Volterra kernels were estimated and used to reconstruct evoked responses to the appearance of a segmented figure from a uniform background. These responses were further analyzed by separate reconstructions of the response to the figure and background regions, as well as their non-linear interaction. The lateral occipital complex (LOC), identified on the basis of an intact vs scrambled object localizer, expressed a strong preference for the figure region, relative to the background, well before 100 msec. At this time, responses in V1-3 were similar for the two regions. An enhanced figure response was observed in early visual areas between 250 and 350 msec, consistent with feedback activation after segmentation by higher-order areas such as the LOC. Figural activity was seen in anterior infero-temporal and frontal cortical areas as early as 100 msec.