Vernier alignment thresholds are strongly compromised when the vernier is embedded in an array of equal-length flanking lines. Here, we show that these contextual interactions can be diminished by giving the flanks the opposite contrast polarity, e.g., white flanks surrounding a black vernier. Similar results are obtained for red verniers and equiluminant green flanks and when vernier and flanks have different binocular disparity. Using special flank configurations, we can eliminate location uncertainty as an important factor for this kind of contextual interactions. We interpret these results as evidence that perceptual grouping of the vernier and the flanks plays an important role in the vernier threshold elevation caused by contextual flanks.