Previous studies of the motion aperture problem have shown that the direction of grating motion can be biased by using binocular disparity to designate borders of the aperture as intrinsic (belonging to the grating) or extrinsic (resulting from occlusion of the grating). Observers report motion in the direction of the extrinsic border, as if the grating was extended and moving underneath an occluding surface. Here we investigate whether prior information about depth ordering, given by structure-from-motion, can bias the perceived motion direction of a subsequent moving grating in a similar manner. We presented an aperture stimulus that rotated about its vertical and horizontal axes, revealing the depth relationships (intrinsic and extrinsic) of the aperture borders. The grating then translated within the aperture and observers reported the direction of perceived motion. The test stimulus contained no information about the depth ordering of the scene. We found that observers’ reported motion shifted toward the direction of the occluding edges, consistent with the intrinsic-extrinsic border predictions. These results indicate that prior scene information, not just depth information explicitly defined in the test stimulus, is used to help solve the motion aperture problem.