Journal of Vision - Luminance–color correlation is not used to estimate the color of the illumination, by Granzier, Brenner, Cornelissen, & Smeets

Jeroen J. M. Granzier	Department of Neuroscience, Erasmus MC, Rotterdam, The Netherlands
Eli Brenner	Department of Neuroscience, Erasmus MC, Rotterdam, The Netherlands
Frans W. Cornelissen	Laboratory for Experimental Ophthalmology & BCN Neuro-imaging Center, School for Behavioural and Cognitive Neurosciences, University of Groningen, The Netherlands
Jeroen B. J. Smeets	Department of Neuroscience, Erasmus MC, Rotterdam, The Netherlands

Humans can identify the colors of objects fairly consistently, despite considerable variations in the spectral composition of the illumination. It has been suggested that the correlation between luminance and color within a scene helps to disentangle the influences of illumination and reflectance, because the surfaces that reflect the light of the illuminant well will normally be bright. Because the reliability of the luminance-color correlation as an indicator of the chromaticity of the illuminant depends on the number of surfaces that are considered, we expected the correlation to be determined across large parts of the scene. To examine whether this is so, we compared different scenes with matched luminance and chromaticity, but in which the correlation between luminance and chromaticity was manipulated locally. Our results confirm that there is a bias in perceived color away from the chromaticity of bright surfaces. However, the results show that only the correlation within about 1° of the target is relevant. Thus, it is unlikely that the visual system uses the correlation between luminance and color to explicitly determine the chromaticity of the illuminant. Instead, this correlation is presumably implicitly considered in the way that the color contrast at borders is determined.

Our visual system somehow manages to recover surfaces’ spectral reflectances despite the fact that the spectral distribution of the light reaching our eyes is determined just as much by the spectral distribution of the illumination as by the surfaces’ chromatic properties. Without additional knowledge or assumptions, either about the illuminant or about the surfaces’ reflectance, it is impossible to separate the two.

Assumptions about the way in which the visual system disentangles illumination from reflection include the possibility that the average reflectance of the whole scene is grey (Buchsbaum, 1980; but see Brown, 2003) or that

the brightest surface is white (Land & McCann, 1971; but see Linnell & Foster, 2002). Obviously, these assumptions are not always correct, and simple experiments show that they cannot explain human color constancy (also see Kraft & Brainard, 1999).

Recently, Golz and MacLeod (2002) proposed a new, more robust variant of the “brightest surface is white” hypothesis. They suggested that the human visual system does not rely only on the brightest surface in the visual scene (assuming that it is white so that the spectral distribution of the light that it reflects is that of the illumination), but rather relies on the correlation between luminance and color across the whole scene to estimate the color of the illumination. If there are many surfaces in a scene, with a large variety of reflectance properties, then it is reasonable to assume that on average the surfaces that reflect well in the color of the illuminant will be brighter. For instance, if the illuminant is reddish, then the surfaces that reflect red light particularly well (i.e., red surfaces) are likely to be brighter than the surfaces that reflect green light particularly well (i.e., green surfaces), leading to a high correlation between luminance and redness within the scene. Thus, this strategy could help disentangle reflectance properties from biases in the illumination, without placing too much emphasis on a single surface.

Golz and MacLeod (2002) presented subjects with scenes in which there were different amounts of correlation between color and luminance, but that had the same average chromaticity and luminance. They found that a test field had to be redder for it to appear perceptually achromatic when the correlation between luminance and redness was high. This is consistent with subjects interpreting the positive correlation between luminance and redness in terms of there being a reddish illumination. Thus, the perceived color was biased away from the color of the brighter patches in the scene, even if the average chromaticity and luminance was held constant.

Golz and MacLeod (2002) implicitly assumed that the luminance–color correlation is determined for the whole scene, or at least their whole display, because that is what one would expect if this scene statistic is used to determine the chromaticity of the illuminant. In the present study, we examined whether this is really the case. We did so by asking subjects to set the color of a disk in a simple computer generated scene. The scene was divided into fields. Each field was built up of squares with two colors: either bright red and dark green, or bright green and dark red (see Figure 1), equivalent to Golz and MacLeod’s fields with a luminance-color correlation of 1. We varied the size of the field surrounding the target, to examine whether this region was of particular importance. When this “near field” did not cover the whole background, it was always surrounded by a field with an opposite correlation between luminance and color.

Figure 1. The four field configurations and two luminance-color correlations in Experiment 1. The adjustable disk was at the center of a background of red and green squares. The square background could be divided into two fields: a near field consisting of a rim of squares surrounding the adjustable disk, and a far field filling the rest of the background. The rim could fill the whole background or it could extend for 4°, 2°, or 1° from the disk. Within each field, either the red squares were brighter than the green, or vice versa. The fields are named by the bright color of the near field.

Like Golz and MacLeod (2002), we compared conditions in which we ensured that the pairs of field colors give the same space-averaged excitation of each type of cone (we refer to this as the “matched sum” balancing method). However, this meant that the darker field colors were more saturated, because lowering the excitation of one type of cone influences the ratio between the stimulation of different types of cones more strongly than increasing the excitation of the same type of cone by the same amount. Because chromatic induction may take place after cone opponency (Brenner & Cornelissen, 2002), and the correlation between luminance and saturation may also be considered (Gilchrist, 2004), such saturation differences could influence the results. If saturation is important, it is not quite appropriate to match the summed cone excitation. We therefore also included conditions in which we matched the cone ratios between the high luminance colors and the low luminance colors (we refer to this as the “matched ratio” balancing method). Obviously, in this case the average L-cone and M-cone excitation was no longer matched.

Ten subjects took part in the experiment. They had normal color vision as tested with Ishihara color plates (Ishihara, 1969). One subject was the first author. The other subjects were naive to the purpose of the experiment. This research is part of an ongoing research program that has been approved by the local ethics committee.

The stimuli were presented on a high-resolution Sony GDM–F520 Trinitron monitor (39.2 cm × 29.3 cm; 1024 × 768 pixels; 120 Hz; 8 bits per gun) in an otherwise dark room. Subjects sat 100 cm from the screen with their chins and foreheads supported. The influence of various backgrounds on the color appearance of a central disk was determined using the hue-cancellation procedure (Jameson & Hurvich, 1955): We determined the physical stimulus that appears to be a neutral grey within different scenes. The extent to which light from each surface stimulated each of the three cone types was determined on the basis of average relative spectral sensitivity functions of human cones (Pokorny & Smith, 1986, Chapter 8).

The stimulus consisted of a 2-deg radius adjustable disk at the center of a 16-deg × 16-deg square background (Figure 1). The luminance of the adjustable disk was 21 cd/m².

The background consisted of an array of 38 × 38 squares. Each square subtended approximately 42 min of arc. It was either red or green (determined at random for each presentation) and either bright or dark. The background could be divided into two fields, a near field and a far field, where “near” and “far” refer to the distance from the adjustable disk. Within a field, either all green squares were bright and all red squares were dark, or vice versa. There were four different near-field configurations (Figure 1). The near field could either fill the complete background (the “all” configuration), or it could fill a ring of 4°, 2°, or 1° width surrounding the adjustable disk. For the latter three configurations, the higher luminance was correlated with the other color in the far field than in the near field. This meant that if the red squares were brighter in the near field, the green squares were brighter in the far field, and vice versa. We will name the luminance-color correlation by the color of the bright squares in the near field, so a “red is bright” luminance–color correlation means that the red squares in the field near the adjustable disk are bright. All the background squares provided the same S-cone excitation, irrespective of their color and luminance.

There were two different color-balancing methods, the matched ratio method (Figure 2a) and the matched sum method (Figure 2b). For the matched ratio method, there were the same two ratios between the stimulation of L- and M- cones within each field. The two possible ratios between L- and M-cone excitations are represented schematically by the dashed lines in Figure 2a.

Figure 2. Schematic (highly exaggerated) representation of the two balancing methods. Dashed lines represent constant cone excitation ratios. Solid lines connect the two colors of each field: bright red and dark green or bright green and dark red. A. The matched ratio balancing method. The colored circles represent the colors that could be present. The mean luminance and chromaticity (open circles) are not the same for the two possible combinations of color and luminance. B. The matched sum balancing method. The colored squares represent the colors that could be present. The open square represents the mean luminance and chromaticity, which was the same for both combinations of color and luminance (20 cd/m²; x = 0.29, y = 0.30). The open circles and dotted lines show how the bright colors were changed relative to their values for the matched ratio balancing method to achieve this (for further details see Methods of Experiment 1).

For each of these ratios, the bright squares had a 20% higher luminance than the dark ones. Each field consisted of squares with the higher luminance for one of the ratios (colors) and squares with the low luminance for the other ratio (see pairs of points connected by lines in Figure 2). The ratio of the L- and M-cone stimulation was 20% larger for the red squares (shallower dashed line) than for the green squares (steeper dashed line). The space-averaged luminance and chromaticity of the two fields was not the same (open circles).

For the matched sum method (Figure 2b), the sum of the L- and M-cone stimulations within each field was the same (open square). To achieve this, we reduced the stimulation of the L-cone in the bright red squares and of the M-cone in the bright green squares, so that the overall average luminance and chromaticity (20 cd/m²; x = 0.29; y = 0.30, open square in both panels of Figure 2) was the same for the “red is bright” and “green is bright” fields. This decreased the saturation of the bright fields. The mean luminance of the background for the matched ratio balancing method was almost 1% higher than for the matched sum balancing method.

Subjects were asked to set the adjustable disk so that it would appear grey. They could vary its color within a two-dimensional isoluminant color space by moving the computer mouse. Subjects indicated that they were content with the set value by pressing a button. Once they did so, a new stimulus appeared. The initial color of the adjustable disk was determined at random from within the range that they could set. Subjects were not instructed to fixate the adjustable disk, although we expected them to direct their gaze at it most of the time anyway (Cornelissen & Brenner, 1995). After dark adapting for 10 min, each subject made 200 settings: each combination of the 4 field configurations, 2 balancing methods, and 2 luminance-color correlations (red is bright or green is bright), each presented 10 times, except for the all configurations that were presented 20 times. We doubled the number of trials for the all configuration because this was our baseline. All the trials were presented in random order. A new field was generated for each trial.

We first determined the mean L-cone value and the mean S-cone value of each subject’s settings for each of the 16 experimental conditions. Note that there was no need to also examine the M-cones, because the settings were made at a fixed luminance. To obtain a measure of how the luminance-color correlation in the field influenced what was perceived as a grey disk, we calculated the difference between the settings when red is bright and when green is bright in the near background (for each cone). We will refer to such differences as “difference scores.” We calculated difference scores for each balancing method and field configuration. This was done separately for each subject, and separately for the L-cone values and the S-cone values.

For the all configuration, we expected the L-cone excitations that subjects set when green is bright, indicating a greener illumination, to be lower than those set when red is bright, indicating a redder illumination. Thus, we expected a positive difference score. For the configurations with near fields that do not fill the whole background, we expect the difference score to be smaller. As the near field becomes smaller, we expect the difference score to become negative. When the near field decreases to a width of zero, the difference score will reach the same value as in the all field configuration, but with an opposite sign, because it is precisely the same stimulus (but with an opposite assignment of the names to the luminance-color correlations). The all field configuration is equivalent to the configuration that Golz and MacLeod used in their experiments (Golz & MacLeod, 2002). As already mentioned, we used this configuration as a baseline. t tests were used to determine whether the subjects’ difference scores in the all configuration were consistently different from zero. Repeated measures analyses of variance were used to evaluate the influence of the field configuration (1°, 2°, 4°, and all) on the difference scores for each balancing method.

Figure 3 shows the mean L-cone difference scores for the four near-field configurations and the two color-balancing methods. The mean L-cone difference scores for the all baseline configuration show a clear trend in the predicted direction (a positive difference score), but these difference scores were only significant for the matched ratio balancing method [t(9) = 5.53, p < .001]. For the matched ratio balancing method, there was also a significant influence of field size [F(1, 3) = 6.89, p = .001] on the mean L-cone difference scores, but the difference scores did not decrease systematically with decreases in near-field size as we had expected. For the matched sum balancing method, the mean L-cone difference score for the all configuration was positive, but it was not reliably different from zero [t(9) = 1.53, p = .16]. No effect of field configuration was found for the L-cone excitation [F(1, 3) = .43, p = .733]. No significant baseline effects and no effects of near-field configuration were found for the S-cone excitation. We had not expected such effects, because we only varied the L- cone and M-cone stimulation in the background.

Figure 3. Results of Experiment 1. Mean difference scores for the L-cone as a function of near-field size. Filled circles: matched ratio balancing method; filled squares: matched sum balancing method. The data for the all configuration have been reproduced as a 0° near-field configuration, with the sign inverted to reflect that the whole background is now considered to be a far field (open symbols). Error bars show the SE between subjects.

For the uniformly correlated field (all configuration), the difference scores for the L-cones confirm that there is a shift in perceived color away from the chromaticity of the bright surfaces (positive difference scores). This shift in perceived color is in accordance with an assumed illumination that is biased in the direction of the color of the bright surfaces. However, this shift was only significant for the matched ratio balancing method. There was also a significant effect of the field configuration for the matched ratio balancing method, but this effect was not due to a systematic change in the difference scores with near-field size, so it is difficult to interpret (see Figure 3). Remember that for the matched ratio balancing method, the shift in perceived color might be explained by the difference in mean cone excitation between the two backgrounds.

The perceived color also appeared to shift in the direction of the color of the bright surfaces for the matched sum balancing method, but this shift was not significant for the all configuration. Because there was no effect of field configuration for the matched sum balancing condition, we also averaged each subject’s difference scores for the four field configurations to see whether the average difference scores differ significantly from zero. The average difference was indeed significantly different from zero when all field configurations were grouped together [t(9) = 4.726, p = .001].

If the correlation between chromaticity and luminance within the whole scene had been used to estimate the chromaticity of the illuminant, we would have expected the difference scores to be positive for the largest near-field configuration (all) and to decrease to negative values as the near-field configuration decreases in size. The near and far fields would have covered the same surface for a near-field width of 6.3°. Thus, if the luminance-color correlation had been determined for the whole scene, we would have expected negative values for all the near-field configurations except for the all configuration. However, even for the 1° near-field width we see a tendency for positive difference scores (see Figure 3). This suggests that only the luminance-color correlation within the surfaces that are adjacent to the surface of interest may be relevant. However, the fact that the baseline difference score was only significantly different from zero for one of the balancing methods warns us to be a bit cautious with such a conclusion. We therefore decided to repeat the experiment with a more sensitive task and even smaller near-field widths.

The apparatus and procedures were identical to those of Experiment 1. The main difference was that in the new experiment a matching task was used instead of a nulling task. The disadvantage of a matching task is that we need two targets with different fields, so that the overall luminance-color correlation cannot be as high. In fact, we always used symmetrical fields, so that the overall correlation was always zero. Thus, if the impression that we got from Experiment 1 was incorrect, we expect to find no effect at all. The advantage of using a matching task is that the reference color is specified explicitly, which we expected would reduce the variability in the settings. We used pairs of backgrounds, each of which was a slightly narrower version of those of Experiment 1 (see Figure 4). We used the same colors as in Experiment 1. If red was bright in one near field, green was bright in the other near field.

Figure 4. The four field configurations and two luminance-color correlations in Experiment 2. Subjects had to set the adjustable disk (on the right) to match the reference disk (on the left). Each half of the background was similar to that in Experiment 1 (for details, see Figure 1). The fields are named by the bright color of the near field surrounding the adjustable disk (i.e., on the right).

If only the luminance-color correlation near the target is important for the perceived target color, as is suggested by the results of Experiment 1, the influence of the correlation could be twice as large here, because the two targets (reference disk and adjustable disk) are each influenced, but in opposite directions. However, we realize that the influence does not need to be exactly twice as large, because there will be differences in viewing strategies between the two tasks, which may influence the color settings that people make (Cornelissen & Brenner, 1991, 1995). In a matching task, subjects move their eyes from the test to the adjustable disk, ensuring that a comparison can be made with the eyes in an almost identical state of adaptation. Thus, changes in adaptation will not necessarily influence the settings. In a nulling task, subjects fixate on the adjustable disk. Because adaptation will not change the remembered reference (in our case grey), it is likely to influence the settings.

Eleven subjects with normal color vision took part in the experiment. Eight of the subjects had also participated in the first experiment, including the first author. Other than the author, none of the subjects knew the purpose of the experiment.

A grey (CIE x=0.29, y= 0.30) reference disk with a luminance of 21 cd/m² was presented at the center of the left background. The disk had the same radius as the disk used in Experiment 1 (2 deg) and was centered on an 11-deg (width) × 16-deg (height) background (see Figure 4). The observer’s task was to match its appearance by manipulating the chromaticity of an equally sized adjustable disk of the same luminance that was presented on an equally sized background on the right. The color of the latter disk could be set within a two-dimensional isoluminant color space by moving a computer mouse.

The fields on the left and right each consisted of an array of 25 × 38 squares. Each square subtended approximately 42 min of arc. The same colors of the field squares were used as in Experiment 1. Again, we had a matched sum and a matched ratio balancing method, with either the red or the green squares being brighter in the near field of the adjustable disk (on the right). We name the luminance-color correlations by the condition in this field (see Figure 4). If the near field of the adjustable disk had bright red squares, then the near field of the reference disk (on the left) had bright green squares, and vice versa. For the far fields, we used the reversed luminance-color correlation that we used in the corresponding near fields. All the near-field widths were halved, so that we now had near-field widths of 0.5°, 1°, and 2°, besides the near field that filled the whole background on each side (all configuration). Thus, once again there were 16 different conditions (4 different field configurations, 2 balancing methods, and 2 luminance-color correlations). Again, the all configuration was treated as the baseline condition.

After dark-adapting for 10 min, each subject made 200 settings: 16 conditions, each presented 10 times with an additional 10 trials in the 4 baseline conditions (all configuration). The 200 trials were presented in random order.

The data analysis was similar to that of Experiment 1. The difference score was now defined as the difference between the adjustable disk’s settings when the bright squares in the field near the adjustable disk were red (red is bright) and when the bright squares near the adjustable disk were green (green is bright).

Figure 5 shows the mean difference scores for the L-cones, as a function of the near-field configuration, for both balancing methods. One-sample t tests showed that the luminance-color correlation had an influence on the L-cone difference scores in the all configurations, for both the matched sum [t(9) = 2.87, p = .017] and the matched ratio balancing method [t(9) = 2.66, p = .024). There were no significant main effects of field configuration for either the matched sum balancing method [F(1, 3) = 1.99, p = .135] or the matched ratio balancing method [F(1, 3) = .46, p = .714]. Again, there were no significant effects for the S-cone difference scores.

Figure 5. Results of Experiment 2. Mean difference scores for the L-cone as a function of near-field size. Filled circles: matched ratio balancing method; filled squares: matched sum balancing method. The data for the all configuration have been reproduced as a 0º near-field configuration, with the sign inverted to reflect that the whole background is now considered to be a far field (open symbols). Error bars show the SE between subjects.

Experiment 2 confirms that the influence of the luminance-color correlation is a local effect. The strongest evidence for this is the fact that the effect is seen when two backgrounds with opposite luminance-color correlations are present in the scene, as was the case in all our displays in Experiment 2. The fact that the difference score is almost the same for a 0.5° near-field configuration as for the largest configuration tested (all), suggests that the effect is limited to the border of the adjustable disk.

We found that the luminance-color correlation had an influence on the L-cone difference scores in all configurations. This finding is consistent with that of Golz and MacLeod (2002), who used equivalent experimental conditions. However, our results suggest that Golz and MacLeod (2002) were incorrect in their implicit assumptions that the visual system uses the correlation between luminance and color in the whole scene to derive the chromaticity of the illuminant. For the luminance-color correlation to provide reliable data for estimating the chromaticity of the illuminant (and, thereby, to separate surface properties from those of the illumination), it is crucial that not just a small part of the visual field is considered, because otherwise the colors of objects which happen to be within the relevant part (e.g., next to the object of interest) will dominate the perceived color (Brenner & Cornelissen, 1991).

We found that extending the color-luminance correlation beyond 1 deg of the test disk had little effect on color appearance. This spatial property is consistent with the spatial properties of chromatic induction (Walraven, 1973; Tiplitz-Blackwell & Buchsbaum, 1988; Brenner & Cornelissen, 1991). This raises the possibility that the present findings and those of Golz and MacLeod (2002) are the result of an interaction between color and luminance when the border contrast is determined. Asymmetries between the chromatic influences of brighter and darker background surfaces have been found before (e.g., Delahunt & Brainard, 2000; Bauml, 2001; Delahunt & Brainard, 2004). In our case, we always have both brighter and darker squares next to the target. However, if the squares that have a higher luminance have a stronger influence on the perceived color, and the effects of all the surrounding squares are additive (Brenner, Cornelissen, & Nuboer, 1989), the summed effect will depend on which color was brighter. Such an asymmetry could explain our data. Moreover, it provides a way to use the ideas underlying Goltz and MacLeod’s proposal for a modest contribution to color constancy without assuming that the illumination is uniform (which it seldom is in daily life).

The overall pattern of the difference scores for the two color-balancing methods was the same. This is not very surprising considering that the difference was extremely small, but it ensures us that the influence that we found is not just a consequence of having equated the fields at the wrong stage of processing. At least, our findings hold whether one equates the fields at the cone (matched sum balancing method) or at the color-opponent (matched ratio balancing method) stages of processing.

In conclusion, while we agree with Golz and Mac-Leod (2002) that there is a bias in chromatic induction away from the color of bright surfaces, we show that this bias is not used, as they implicitly suggest, to estimate the chromaticity of the illuminant from the correlation between luminance and chromaticity within the whole scene.

This work was supported by grant 051.02.080 of the Cognition Program of the Netherlands Organization for Scientific Research (NWO).

Commercial relationships: none.

Corresponding author: Jeroen Granzier.

Email: j.granzier@erasmusmc.nl.

Address: Department of Neuroscience, Erasmus MC, P.O. Box 1738, 3000 DR, Rotterdam, The Netherlands.

Cornelissen, F. W., & Brenner, E. (1991). On the role and nature of adaptation in chromatic induction. In B. Blum (Ed.), Channels in the visual nervous system; neurophysiology, psychophysics and models (pp. 109-123). London and Tel Aviv, Freund Publishing House.