Journal of Vision - Depth information affects judgment of the surface-color mode appearance, by Yamauchi & Uchikawa

The mode of color appearance is determined not solely by physical properties of the stimulus but also by the conditions of surrounding stimuli. Coplanar ratio hypothesis suggests that the information provided in the same plane plays an important role in the judgment of lightness. We measured the upper-limit luminances of the test stimulus for the surface-color mode in a three-dimensionally represented environment to study the effects of depth and luminance conditions on the mode perception. The test stimulus and two array-type surrounding stimuli composed of 10 different colors were presented at different depths. The test stimulus was presented at three different depths. Subjects set the luminance of the test color to the point where it just ceased to appear in the complete surface-color mode. The upper-limit luminances of the test colors varied as the luminances of the surrounding stimulus displayed in the same depth changed. Our results indicate that the perception of the surface-color mode is mainly determined by the stimulus displayed in the same depth. These results support that belongingness – to which group in the environment the stimulus belongs – is important, and that the mode of color appearance is determined coplanar in a three-dimensional environment.

When an illuminated surface is observed through a small window, the perceived color through the window changes dramatically as the intensity of the illumination increases. When the intensity of the illumination is low enough, the color appears opaque as if the window itself is a surface. As the intensity of the illumination increases, the color appears brighter. Then there is a point at which the color starts to appear fluorescent, followed by the appearance of the light source as if a light is being emitted through the window, or the window itself glows, as shown in Figure 1. Katz (1935) explained this transition of color appearance with the term “mode of color appearance.” The transition described above can be explained by the change of the mode of color appearance: The mode of color appearance changed from the surface-color to the aperture-color.

Figure 1. Example of the transition of the mode of appearance. When a brown paper is dimly lit (a), its appearance through a window is opaque, whereas when it is brightly lit (b), it appears as if the window itself glows or the light is emitted through the window.

We have investigated this transition of the mode of appearance by measuring the luminance of the test stimulus when it ceased to appear as a complete surface (e.g.,Yamauchi & Uchikawa, 2000; Uchikawa, Koida, Meguro, Yamauchi, & Kuriki, 2001). In this study, we refer to this luminance as “the upper-limit luminance for the surface-color mode appearance.”

There have been many studies trying to clarify the transition of the mode of appearance. Evans (1959) defined “G₀ color” as the point at which no grayness is perceived. This is substantially equivalent to the transition point from the surface-color to the aperture-color mode. Evans and Swenholt (1967) measured G₀ color for a range of colored light with several purities, and reported its wavelength dependence.

Bonato and Gilchrist (1994) defined “luminosity threshold” as the point at which the stimulus starts to appear luminous. This is closely related to the transition point from the surface-color to aperture-color mode. They reported that the luminosity threshold of the achromatic test stimulus was about 1.8 times higher than that of the surrounds. Using chromatic stimuli, Speigle and Brainard (1996) reported that luminosity thresholds are different depending on the color. Yamauchi and Uchikawa (2000) measured the upper-limit luminance for the surface-color mode, which is close to the luminosity threshold. They found that the perceived brightness, not luminance, was almost the same for all chromaticities tested. Moreover, Uchikawa et al. (2001) reported that this criterion showed the same wavelength dependence as that of brightness matching, which supports the results reported by Yamauchi and Uchikawa (2000).

To explain this phenomenon, several theories have been proposed, such as an anchoring theory (Gilchrist & Bonato, 1995; Gilchrist et al., 1999) and the highest luminance ratio hypothesis (Wallach, 1948). They deal primarily with how lightness is evaluated. The anchoring theory explains that the visual system sets an anchor for lightness scaling, and the lightness of the surface is judged based on this anchor. When the stimulus exceeds the scale of the surface, it appears luminous. On the other hand, the highest luminance ratio hypothesis explains that white works as the anchor for lightness judgment because white is the brightest surface in the scene. Ikeda and his colleagues have been trying to explain the phenomena with the term “recognized visual space of illumination” (e.g., Ikeda, Shinoda, & Mizokami, 1998). This notion is based on the idea that we easily recognize how bright the surface in the scene can be by estimating the intensity of the illumination.

Moreover, the importance of the organization of the stimulus has been pointed out by several researchers (e.g., Gilchrist, 1977; Adelson, 1993; Agostini & Proffitt, 1993). Gilchrist (1977) showed that the perceived lightness changed dramatically depending on the location of the stimulus, which is explained by the “coplanar ratio hypothesis.” Bonato and Cataliotti (2000) also pointed out that perceptual organization is an important clue for judgment of the lightness of the stimulus.

These findings are not restricted to two-dimensionally presented stimulus. There have been many studies that have dealt with a stimulus being presented three dimensionally (e.g., Gilchrist, 1977; Schirillo, Reeves, & Arend, 1990; Schirillo & Shevell, 1993; Perkins & Schirillo, 2003). Some studies used a real experimental room, whereas others used a stereoscopic apparatus for the subject to fuse an image to form a three-dimensional (3D) image. Schirillo and Shevell (1993) measured perceived lightness and brightness of an achromatic surface presented three dimensionally, and showed that those judgments are influenced by the luminance that shared the same depth plane. Ikeda et al. (1998) insisted that it should be considered “co-spatial” rather than “co-planar.” In addition, phenomena affecting lightness perception, such as the orientation of the surface and the pose, have also recently been investigated (Boyaci, Maloney, & Hersh, 2003; Ripamonti et al., 2004).

To clarify how the mode of appearance is judged in natural settings, we need to expand our experimental setting to include 3D, so we can see how surface-color mode perception is affected by depth information and to check whether our previous findings still work in this condition. The purpose of this study is to clarify the effects of depth information on the judgment of the limit for the surface-color mode perception. We used stereoscopic stimuli that simulated colored papers in a virtual 3D environment.

The experimental booth consisted of two small rooms, one for the stimulus presentation and the other for an observer to sit in. A shutter was placed to cover the window through which the observer viewed the stimulus. The experiment was conducted with a single computer-controlled CRT monitor. The observer booth was lit with a D₆₅ simulating fluorescent lamp to prevent dark adaptation. The monitor was split into two areas with a black-painted cardboard to deliver the images only to the right or left eye. Observers fused the images presented to each eye through a stereoscope to perceive a stereoscopic image with depth. The luminance and chromaticity of the stimuli were carefully calibrated. The viewing distance was set at 100 cm. The head position of the observer was steadied with a chin rest. The luminance of the test stimulus was variable and controlled with a trackball, which was connected to the computer.

We simulated several configurations of the stimulus. The stimuli consisted of the background, two surrounding stimuli, and a test stimulus. A schematic diagram of perceived fused CRT images is shown in Figure 2. Figure 2(a) and 2(b) depict the example of the room- and the plane-type background configurations, respectively. Details of the stimuli are described below.

Two types of background configuration were used: room-type and plane-type configurations. In the room-type configuration, the distant wall subtended 9 by 6 deg, while the frontmost area subtended 13 by 9 deg. The luminance of the walls decreased as the depth increased. Each corresponding vertex was connected, so they provided a strong cue of the parse to strengthen the depth perception.

Plane-type configuration had only a single depth. The plane consisted of two rectangles of the different luminances: the luminance of the center was set to be identical with that of the distant wall in the room-type to ensure that the luminance around the test stimulus was identical. The luminance of the surrounds equaled the mean luminance of the side walls in the room-type configuration.

The surrounding stimulus consisted of two colored squares displayed at a different depth as shown in Figure 3. Each surrounding stimulus subtended 4.5 deg and consisted of 6 by 6 square color chips. Each color chip was a 0.75-deg square, and 10 different colors were used. The distribution of the color chips was identical for both squares. The color chips were placed randomly but none of the same color chips were located next to each other. The luminances and chromaticities of the stimuli used in the experiment are listed in Table 1. Two luminance levels were adopted under the same chromaticities. The luminances listed in Table 1 served as standard, and will be referred to as 100%. For the other luminance level, the luminances were set to be half of the standard, and will be referred to as 50%. The mean luminance of the surrounding stimulus in the standard condition was 12.6 cd/m². The disparity was set so the upper stimulus appeared closer to the observer than the lower one. The difference in the disparity between these two surrounds was 48 arcmin.

We investigated two conditions for the relative position of the surrounding stimuli and the test stimulus: the adjacent-condition and the gap-condition. In the adjacent-condition, the test stimulus and the surrounding stimuli are located next to each other, by sharing an edge, while in the gap-condition, a gap of 0.75 deg was introduced between the edges of the surrounding and the test stimulus. Because of the size constraint of the monitor, we could not use the same configuration of the surrounding stimuli in both conditions. We changed the configuration of the array from a square consisting of 6 by 6 color chips to a rectangle of 8 by 4 color chips.

The observer adapted to the D₆₅ simulating fluorescent lamp for 3 min before each experimental session started. He then opened the shutter to observe the stimulus through the window. After fusing the images, the observer reported what the stimulus looked like, and how all the stimuli were illuminated. The observer was also asked whether any stimulus in the scene appeared luminous. Then the experimental sessions started. In each trial, he adjusted the luminance of the test color so that it just started to appear as an aperture color. The surrounding stimuli always appeared as a paper surface. The instruction was to set the luminance of the test stimulus at the level perceived to be the limit of surface-color mode.

When an adjustment was completed, the observer pressed a button on the trackball. The next trial started after a 2-s blank interval. A session was composed of 40 trials, in which eight different test colors were presented in five different positions in random order. The spatial configuration of the stimulus and the luminance settings of the surrounding stimuli were kept constant within a session. We conducted five sessions for each condition. The observer was instructed to pay attention to the entire stimulus while adjusting the luminance of test stimulus.

Four observers (three males and one female) with normal color vision and normal or corrected visual acuity participated in the experiments. They were naïve to the design and the purpose of the experiments, except for YY, who was one of the authors. Each participant’s color vision was tested with Ishihara plates and a Farnsworth-Munsell 100-hue test. They had previous experience participating in similar psychophysical experiments conducted with the same criterion.

Experimental results obtained in the adjacent-condition are shown in Figures 5 and 6. Figures 5 and 6 show the upper-limit luminances for the surface-color mode obtained from one observer (YY), and the mean luminances across all the observers, respectively. The abscissa indicates a test color number, as defined in Figure 4. In Figure 5, (a) and (b) indicate the results obtained when the test stimulus was presented at the same depth as the frontal surrounding stimuli in the room- and plane-type configurations, respectively. Error bars shown in the panels indicate the standard deviations. All observers had similar standard deviations. Four panels in Figure 6 show the results obtained in front, room (a) (which means the test stimulus was presented at the same depth as the frontal surrounding stimuli in the room-type configuration); rear, room (b); front, plane (c); and rear, plane (d). In each panel, the solid black circle, solid blue triangle, and open red square symbols denote the luminance conditions of the surrounding stimuli to be 100/100%, 100/50%, and 50/100%, respectively. The solid orange diamond and open green triangle symbols denote the results obtained when the test stimulus was presented inside the surrounding stimulus of 100% and 50%, respectively.

Figure 5. The upper-limit luminances obtained from subject YY in the adjacent-condition in the room-type (a) and the plane-type (b) configurations. The test stimulus was displayed in the front depth. The abscissa denotes the stimulus number defined in Figure 4. The solid black circle, solid blue triangle, and open red square symbols denote the luminance conditions of the surrounding stimuli to be 100/100%, 100/50%, and 50/100%, respectively. The solid orange diamond and open green triangle symbols denote the results obtained when the test stimulus was presented inside the surrounding stimulus of 100% and 50%, respectively.

As shown in each figure, the upper-limit luminances for the surface-color mode changed depending on the luminances of the surrounding stimulus displayed in the same depth. The upper-limit luminances of the test stimulus presented in the front plane were higher in 100/50% condition than in 50/100%. When the test stimulus was presented at the rear position, on the other hand, the upper-limit luminances of the test stimulus were higher in 50/100% condition than in 100/50% condition.

In either configuration, the upper-limit luminances obtained when the test stimulus was presented between the surrounding stimuli were lower than those when the test stimulus was presented inside the surrounding stimulus. The amount of color information displayed to the observer was the same, but the spatial configuration of the stimulus was different: The test stimulus was adjacent to two (or four) color chips in the adjacent-condition, while it was surrounded by 12 color chips when it was displayed inside the surrounding stimulus. Thus the chromatic contrast might play a role in causing such differences. We will refer to this point later in the General discussion.

To rule out the possibility that merely the spatial position and the relative luminances led to these results, we conducted two supplementary experiments. First, we swapped the depth of these two surrounding stimuli, locating the lower surrounding stimuli closer to the observer than the upper one. Our results were the same in this condition.

Second, we conducted the same experiment but without depth. Two surrounding stimuli with the interval of 1.5 deg were displayed on the plane-type background. The test stimulus was displayed in one of three positions: inside the upper surrounding stimulus, inside the lower surrounding stimulus, and between the two. The luminance settings of the surrounds were also the same as in the first experiment. The upper-limit luminances for the surface-color mode were almost the same for all three positions in 100/100%. The luminances of the test stimulus were significantly lower when it was displayed inside the darker (50%) stimulus. When two surrounding stimuli had different luminances, the results obtained from the stimulus between two surrounding stimuli were somewhere between those obtained in 100% and 50%. In addition, the luminance change was smaller compared to that obtained from the test stimulus displayed at different depths.

The upper-limit luminances were different among colors. The chromatic characteristics are quite similar to those obtained in previous experiments (Yamauchi & Uchikawa, 2000, 2004b, 2004c). The general trend is that the more saturated the test color, the lower the upper-limit luminance. There were some individual differences in the luminance, but all of them showed similar trends. When the luminances of each test color are multiplied by B/L values for that color to convert the luminance to brightness, the differences in brightness among test colors were much smaller, as was reported previously (Yamauchi & Uchikawa, 2000).

The upper-limit luminances for the surface-color mode tended to be lower in this experiment than those obtained in the adjacent-condition, especially in plane-type configuration. All four observers showed the same trends as in the adjacent-condition, and the luminances changed in the same way as the luminance change of the surrounding stimulus at the same depth.

In Experiment 1, we found that the upper-limit luminances for surface-color mode appearance depended on the luminance condition of the stimuli at the same depth. As the observers could easily find that those stimuli were located at the same depth, they might use the information provided by the surrounding stimulus for their judgment.

Then, what if there is no explicit information about the location of the test stimulus? That is, there is no surrounding stimulus that shares the same depth as the test stimulus, but the test stimulus is located between two surfaces of different depth. If observers can estimate the overall conditions based on the provided information, they may be able to interpolate the surface-color mode perception.

The results obtained in the adjacent-condition are shown in Figure 8. (a) and (b) indicate the mean luminances across four observers obtained in the room- and the plane-type configuration, respectively. The symbols are the same as those used in Figure 6. For reference, the results obtained when the test stimulus was presented inside the surrounding stimuli were plotted in the same panel. It is shown that the results were similar to those obtained in Experiment 1. The results obtained in the gap-condition are not shown in the figure, but it showed the same trends as those in the adjacent-condition.

Next we analyzed how much the upper-limit luminances changed with a change in the surrounds’ luminance. To achieve normalization, the “standard luminance,” the upper-limit luminance obtained with the test stimulus displayed in 100/100% without depth, was used. The normalized values were averaged across eight test colors to represent each condition.

The normalized luminances are shown in Figure 9 for both the room- and the plane-type configurations. (a) and (b) indicate the values obtained from the room-type and the plane-type background, respectively. They are the averaged values over the four observers. Error bars indicate ±1 SE. The different symbols denote the different luminance settings of the surrounds: The black circle, green triangle, and red square symbols indicate the results of 100/100%, 100/50%, and 50/100%, respectively. The solid symbols and open symbols denote the values obtained from the adjacent- and the gap-condition, respectively. The orange diamond and blue triangle symbols that are shown in the leftmost and the rightmost position in the figures denote the normalized luminance when the test stimulus was displayed inside the surrounding stimuli, whose luminance was 100% and 50%, respectively. The leftmost and rightmost symbols denote those results obtained in the front and the rear, respectively.

Figure 9. Mean normalized luminance across observers in room-type (a) and plane-type (b) background for both the adjacent- and the gap-condition. The black circle, green triangle, and red square symbols indicate the results of 100/100%, 100/50%, and 50/100%, respectively. The orange diamond and blue triangle symbols indicate the values obtained when the test stimulus was presented inside the 100% and 50% surrounding stimuli, respectively. The solid symbols and open symbols denote the values obtained from the adjacent- and the gap-condition, respectively.

In Figure 9, it is clearly shown that the results obtained in Experiment 2 were just between those obtained in the front and rear conditions, especially in the conditions when the two surrounding stimuli had different luminances (50/100% and 100/50%). These results confirm our hypothesis that observers can correctly interpolate the judgment for the surface-color mode based on the sparse information provided three dimensionally.

To find and compare trends among the conditions, we compared the slope of each result by line fit. It turned out that most of the results fit well with a linear equation (R²> 0.98), except for the 50/100% adjacent-condition in the room-type configuration, and 100/100% both adjacent- and gap-condition in the plane-type configuration. All slopes differed significantly from a flat line (α < .01). We used these slopes to compare different conditions. The slopes σ² and R² are listed in Table 2.

The good fit with a linear equation indicates that the observers could interpolate the criterion for the judgment of the surface-color mode precisely based on some information that could be used as a clue. The reason why 100/100% in the plane-type configuration did not fit into a linear equation was probably because the clues included in the stimuli did not establish a rigid criterion.

The comparison among slopes in the same configurations revealed that all combinations (e.g., 100/100% vs. 100/50% in the room-type) were significantly different (α < .01) in both adjacent- and gap-condition. The slope of the luminance gradient of the wall in the room-type configuration was –0.0103, which was much larger than those of 100/100% and 100/50%. Because the configuration was the same, these significant differences came from differences in the surrounding stimuli. This means that the luminance conditions of the surrounding stimuli were the primary determinant for the limit on the surface-color mode perception.

Although the slopes of 100/100% in the room- and plane-type configurations are significantly different (α < .01), other corresponding conditions (100/50% plane vs. 100/50% room and 50/100% room vs. 50/100% plane) are not significantly different (α = .05). As for the 100/100% condition, all values were almost the same in the plane-type configuration, whereas in the room-type configuration, the values decreased as the test stimulus was presented further. This means that the luminance gradient in the room-type configuration worked as a cue when the surrounding stimuli failed to provide an explicit cue. On the other hand, in the plane-type configuration, there was no help in judging the global environment. When two surrounding stimuli had any difference in luminance, both configurations showed the same trends. This means that local information is the primary factor.

Among the pairs of the same conditions in the room- and plane-type configurations, there is only one condition that has remarkable differences: 50/100% in the room-type condition. The normalized luminance change did not increase in the rear condition as in the plane condition. Instead, the normalized luminance was almost the same as that obtained at the intermediate depth. This difference might be because the global luminance information provided by the background and the local luminance information given by two surrounds did not match. This is not clear when the same luminance patterns are displayed with a spatial gap. Cataliotti and Gilchrist (1995) reported that both local and global information affect lightness perception. Our results support this notion.

Cues provided to the observers were only the surrounding stimuli in the plane condition, whereas in the room-type condition, the luminance gradient set on the wall might have strengthened the effects of the luminance change because the front surrounds were higher than the rear surrounds. Having relatively more information from the surrounding stimulus works in interpreting the environment in the plane-type, whereas in the room-type, the luminance gradient might work as a main factor.

Then how far can the gap between the test stimulus and the surrounding stimuli be while maintaining the influence of the surrounding setting on the test stimulus? If distance has nothing to do with perception, any information that is displayed inside the visual field can affect perception. This hypothesis, though, is not plausible. As is shown in Figure 9, luminance decreased when a gap was introduced between the test and the surrounds. This is discussed from the viewpoint of the anchoring theory in the next section.

In anchoring theory, the anchor that serves as a criterion for lightness scaling is set for a scene. Several clues may exist to find an anchor. Moreover, in a natural scene, the number of anchors is not necessarily limited to one. Gilchrist et al. (1999) referred to the range that a single anchor can hold as a “sub-frame.” In this sense, the anchor can be set for each sub-frame. Our experimental results showed that the upper-limit luminances for the surface-color mode changed significantly depending on the luminance condition of the stimulus displayed at the same depth as the test stimulus.

If the same anchor can be applied within a sub-frame, it is important to know how observers find the range over which a sub-frame is applicable. Here it becomes important to know how to understand the organization of the environment. Once we know to which group an area belongs, we can apply the sub-frame for that group in judgment. Bonato and Cataliotti (2000) pointed out the importance of organization for lightness judgment; depth information is an important factor that helps in the judgment of the mode of appearance.

Considering that the plane-type and room-type configuration behaved differently in the 50/100% condition, a different sub-frame might be used in those two cases, even if the surrounding stimuli were identical. The room-type configuration provided an explicit clue to the observers: a luminance gradient in the wall, to infer the nature of the illumination on the entire environment.

When a gap existed between the test stimulus and the surrounding stimuli, the effect of the sub-frame could be weakened. The test stimulus might be rather isolated. Thus, it is plausible that this weakened binding caused by the gap lowered the upper-limit luminances. From the viewpoint of the grouping, the gap between the test and the surround certainly worked to isolate the test stimulus from the surrounding stimulus.

Thus our results strongly support the notion that the perception of the surface-color mode is based on an anchoring theory; also, the perceptual organization of the stimulus, or the grouping of the stimulus, plays an important role in the judgment of the mode of appearance.

It is noteworthy that the observers reported that they felt like an extra illumination existed in 50/100% in the room-type condition. As the instruction for the observers did not refer to its illumination, the observer might interpret the scene in a way that minimizes the contradiction of the luminance pattern. Bonato and Gilchrist (1999) reported that the larger the size of the stimulus, the higher the luminosity threshold. This is probably the case in our settings. When the surrounding stimulus subtended a certain area, it might be easier, or much more natural, to assume an extra illumination for that stimulus or to construct a new sub-frame, rather than perceiving such a wide area to be luminous.

As mentioned above, all the observers showed the same trend: that the stimulus presented inside the surrounds had a higher luminance for the limit of the surface-color mode than that adjacent to it. If the coplanar ratio hypothesis was fully applicable, this should not be the case. Instead, they should be the same. Here we cannot ignore the spatial interaction of the surrounds, such as chromatic induction (e.g., Blackwell & Buchsbaum, 1988) and simultaneous contrast (Arend & Goldstein, 1987). Shevell and Wei (1998) reported that chromatic induction was observed from the remote chromatic contrast. Schirillo and Shevell (1993) reported that the non-uniform configuration of the surrounds affected the brightness contrast of the stimulus. Also, Brown and MacLeod (1997) reported that the color appearance of the center was affected whether the surrounding stimulus was composed of a color-rich stimulus or not. In our experiment, the test stimulus shared its edge with only 2 colors in the adjacent-condition, while it was surrounded by 12 colors in the inside condition. The amount of chromatic contrast was clearly different. Taking mutual interactions into consideration, these differences in the stimulus configuration might affect the judgment of the surface-color mode.

Moreover, when we conducted the brightness matching of the test color and the reference white in various background conditions to obtain B/L ratio, the B/L of all the observers was smaller for the mondrian-type stimulus than for the center-surround stimulus. This result is in keeping with the results reported by Shevell and Wei (1998) that the color contrast contained in the remote area also affected the appearance of the color at the center. This can explain why the upper-limit luminances for surface-color perception were not the same in the two control conditions (the diamond and triangle symbols in Figure 9), although the two surrounding stimuli had the same luminance and the same surroundings. If we had adopted exactly the same configuration for the surrounding stimulus, we might have gotten the same results in both the adjacent- and gap-condition.

As the difference in depth was provided by the disparity, local contrasts between the test and the surrounding stimulus were kept constant in the retinal images. Nevertheless, the upper-limit luminances for the mode of color appearance changed as the perceived depth and the luminance values of the stimulus presented at the same depth. There might be a sub-frame in local contrasts that can affect the appearance.

Our results indicate that the mode of color appearance is judged based on both depth and luminance information. Our results support the idea that perceptual grouping and perceptual belongingness are important for the judgment of luminosity, and the mode of color appearance is determined coplanar in a 3D environment. However, the influence of local information cannot be completely ignored. Further study is required to unveil how sub-frames are constructed based on visual information, including the estimation of illumination.

Ripamonti, C., Bloj, M., Hauck, R., Kiran, K., Greenwald, S., Maloney, S. I., et al. (2004). Measurements of the effect of surface slant on perceived lightness. Journal of Vision, 4(9), 747-763, http://journalofvision.org /4/9/7/, doi:10.1167/4.9.7. [PubMed][Article]