How does chromatic information influence gloss perception? For glossy dielectric surfaces, pixel colors are linear combinations of the colors of specular and diffuse reflectance components. We generated images of glossy corrugated surfaces and independently changed the colors of the two components, while keeping the luminance profile fixed. We used a high-dynamic-range display to obtain sufficiently bright colored highlights. When diffuse and specular components shared the same color (e.g., white on white, or red on red), observers perceived normal glossy surfaces. When the specular component was white and the diffuse component was colored (e.g., white on red), the surface looked even more naturalistic. On the other hand, when we combined a colored specular component and a white diffuse components (e.g., red on white), the surface images looked somewhat strange. They looked less glossy, and more importantly, did not appear to have a uniform reflectance. Colored highlight regions appeared to be spatially segregated from the surrounding white-body regions, as if pieces of colored foil were attached to a white matte surface. Of the four color combinations, only the last one (color on white) contradicts our usual experience with dielectric surfaces. A white object under white light gives white on white; a white object under colored light gives color on color; a colored object under white light gives white on color, but in most circumstances we don't encounter color on white. Our observation suggests that the human visual system correctly takes into account a physical constraint of highlight color when it judges whether a given local luminance change is introduced by superposition of a highlight, or by other physical causes including reflectance changes. Thus white on red gives the best sense of gloss, and red on white gives the worst.