1. Introduction

1.1 Memory color matching (MCM)

Memory color, the color associated with a familiar object in long-term memory, has been used successfully as an internal reference in the study of chromatic adaptation, image color quality and color rendition evaluation. Smet et al. used the memory colors of familiar objects as an internal reference to investigate chromatic adaptation under neutral [1] and colored [2] illumination. Xue et al. [3] illustrated a method using on-screen memory colors for image enhancement. Smet et al. [4–6] presented a color rendition index based on memory colors to evaluate the visual appreciation of white light sources and showed that it outperformed several other measures published in literature.

The memory color matching method involves an observer adjusting the color appearance of a familiar object stimulus under an adaptive condition until it matches the observer’s internal memory color. Note that the type of match that observers are asked to make, color appearance match versus a surface match, can have an impact on the results [7]. When determining corresponding colors, the memory color matching method allows for more accurate matches than short-term ‘learned’ memory matching [8] and is less time consuming than simultaneous asymmetric matching, as it avoids switching back and forth between these conditions. Note that the MCM can provide data at several points in color space compared to the achromatic matching method often used in color constancy studies. An overview of the details, advantages and disadvantages of the method and how it compares to other asymmetric matching methods is presented in Smet et al. [1].

1.2 Previous studies on starting point or anchoring bias

Judgement bias resulting from starting points has also been called anchoring [9]. It can affect a broad range of judgements, including answers to knowledge questions, monetary evaluations, and social judgments [10].

In lighting industry, many studies have been done to investigate the effect of starting point on glare evaluation, and preferred luminance. Kent et al. [11] investigated starting point bias by conducting a Hopkinson-like multiple criterion adjustment experiment with three different initial anchors. The glare source was a diffusing screen illuminated by a projector, with the luminance level ranging from 200 to 32000 cd/m². During the experiment, the researcher adjusted the luminance of the glare source starting from different anchor points and the participants were instructed to indicate when each of the predefined discomfort glare criteria was reached: Just Imperceptible, Just Acceptable, Just Uncomfortable and Just Intolerable. The results showed a significant starting point bias, whereby glare settings were always biased towards the initial luminance of the glare source before the adjustment. Logadottir et al. [12] investigated the effect of pre-adjustment anchoring on the preferred luminance in an office setting. The experiment included three different stimulus ranges and three anchors were used within each range. The participants were instructed to adjust the amount of light in the test room to the levels they preferred in a text-reading task. It was demonstrated that for each range, the anchor had a significant effect on the results: higher anchors resulted in a higher preferred illuminance.

According to the literature about measurements of absolute thresholds by the method of adjustment, the general procedure is to set the stimulus intensity level either far below or far above threshold and then to take the mean of the adjustment values as the final results [13]. Logadottir et al. [12] examined the influence of a mid-range anchor (A2) in addition to lower (A1) and higher anchors (A3). They found that the preferred illuminance estimated as the mean of those using anchors A1 and A3 was significantly lower than that when only using anchor A2, while the estimates using A2 and the mean of A1, A2 and A3 were reasonably similar.

However, the anchor effect in terms of chromaticity has hardly been experimentally demonstrated. Smet et al. [2] adopted a memory color matching method to collect corresponding color sets under background illuminations with different chromaticities. To avoid bias from starting points chromaticity, four colorful starting points, equally distributed along the hue circle centered at the chromaticity of the target object, were used. Zhai et al. [14] asked observers to identify the most neutral patch among 49 NCS color patches under various illuminants. To reduce the bias effect, they started from the center patch (S-0500-N) and verified the decision by letting observers view all the neighboring patches. However, a systematic experimental verification of the influence of the starting point on the end point has, to our knowledge, never been reported. The present study implemented a similar memory color matching experiment as in Smet et al. [2], but with more starting points, to collect corresponding color data under two background illuminations. The goal of the present study is (1) to investigate how the distribution and combination of starting points and the number of participants influence the accuracy of the final match, (2) how the anchor bias can be minimized, and (3) how a practical trade-off between accuracy and experiment duration can be achieved.

2. Experiment design

2.1 Apparatus

Memory colors were collected for three objects: a neutral grey cube, a yellow lemon and a green apple, presented on a white background (diffusely reflecting white paper) under two adapting illuminations. The rationale for selecting a green apple and a yellow lemon is that they are considered as familiar objects. Furthermore, neutral grey is often used as a target object color in color constancy studies adopting an achromatic matching method. The background, with an adapting luminance of approximately 600 cd/m², was chosen to represent a neutral white light source with equal energy white (EEW) chromaticity coordinates, as well as a highly colorful yellow light source. Note that the neutral white background is only a metameric match to EEW, i.e., it has the same chromaticity coordinates, but a different spectrum. Each memory color match was repeated 10 times, always starting from a different chromaticity. Ten starting points were selected: 8 colorful starting points (SP) equally distributed along a hue circle (in the CIE 1976 u’₁₀ v’₁₀ chromaticity diagram) centered at the familiar object chromaticity when illuminated by neutral white (SP1 - 4, SP6 - 9), one at the familiar object chromaticity corresponding to an EEW illumination (SP5) and finally one corresponding to the object chromaticity when illuminated by the background illumination (SP10). Note that under neutral white (EEW) illumination, SP10 is identical to SP5 for all the objects and there are only nine different starting points. The chromaticity distribution of the two illuminations and ten starting points in u’₁₀ v’₁₀ space for each object are shown in Fig. 1. The eight highly chromatic starting points, close to the edge of the projector color gamut, were chosen to maximize the size of the ellipse. The starting point chromaticity for all three objects was realized using the same projector illumination settings, thus resulting in different actual starting point chromaticities for each object due to their specific reflectance factors.

 

Fig. 1. The distribution of the starting points SP1-SP4, SP6-SP9 (black spots) & SP5 (a red square) and the two backgrounds (blue stars) in the CIE 1976 u’₁₀ v’₁₀ chromaticity diagram. SP10 is not plotted here because it overlaps with SP5 under EEW background and is very close to SP1 under yellow background. The picture of each object is shown in the upper right corner. (a) Grey cube (b) Green apple (c) Yellow lemon

Download Full Size | PDF

As illustrated in Fig. 2(a), the background scene, with a field of view (FOV) of approximately 50°, was composed of white paper (with minor fluorescence) and contained various neutral 3D objects to increase the scene realism and to provide depth, which have been reported to influence color constancy and chromatic adaptation [15].

 

Fig. 2. The experiment setup. (a) Picture. (b) Side view schematic.

Download Full Size | PDF

A similar experimental setup was adopted as in Smet et al. [16,17], whereby a calibrated data projector was used as a simple spatially tunable light source, allowing independent adjustment of stimulus and background illumination. The whole scene is illuminated by the same data projector, where the color of each pixel can be controlled independently. Before the experiment, the object shape and the projected pixels of the data projector had been carefully characterized, such that it is known which pixels of the data projector illuminate the object and which pixels illuminate the background. The RGB values of the pixels illuminating the background were fixed (but different depending on the background setting), while the RGB values of the pixels illuminating the object could be adjusted by the participant. As shown in Fig. 2(b), the test stimulus was a 3D grey (spectrally flat, see Fig. 3(a)) cube with a 6° FOV and centrally positioned in the background scene. Independent control of the background illumination color and the test objects color with various shapes and sizes could be easily achieved by the use of a mirror underneath the test object (see Fig. 2(b)). It must be noted that the mirror was invisible from the observers’ viewpoint. The reflectance spectra of the background (white paper) and the objects (grey cube, green apple, yellow lemon), as measured by a Hunterlab UltraScan Pro colorimeter, are plotted in Fig. 3(a). The spectral radiance of the background and the objects was measured with a calibrated OceanOptics QE65Pro tele-spectroradiometer at a 2 m measuring distance. The radiance spectra of both backgrounds are plotted in Fig. 3(b). The background chromaticity is iteratively optimized at the start of each experiment day to have a u’₁₀v’₁₀ distance to the target chromaticity smaller than 0.001. As the object size was larger than 4°, all colorimetric calculations were done using the CIE 1964 10° color matching functions. By only changing the color of the light illuminating the object, the perceived color of the object could be changed, allowing observers to make their memory color adjustments.

 

Fig. 3. (a) The reflectance spectra of the grey cube, green apple, yellow lemon (stimulus) and white background used in the experiment. (b) The radiance spectrum of two backgrounds (EEW and yellow).

Download Full Size | PDF

2.2 Methods

Observers were asked to adjust the color of the presented familiar object until it matches their memory color. Observers could navigate in the CIE 1976 u’₁₀v’₁₀ space using the 4 arrow keys (up, down, left, right) of a regular keyboard. Note that while changing the object chromaticity, its luminance was maintained at around 200 cd/m², 330 cd/m² and 280 cd/m² for the grey cube, yellow lemon and green apple, respectively. As indicated before, the background luminance was always around 600 cd/m². To ensure the observers’ adaptation state is sufficiently steady [16,17], they could only start making their match under that background chromaticity after an adaptation time of 45 seconds. By the time they had finished a match, more than a minute had passed and they would have reached more than 90% of their final steady-state adaptation state [16]. The backgrounds (EEW and Yellow) and starting points were presented randomly. After each satisfactory match the spectrum of the match was recorded and the illumination changed to the next situation.

2.3 Observers

All participants in this study were students, recruited from the University. They were required to have normal color vision, which was tested with the Ishihara 24-plate test. For the grey cube, 10 observers (7 males and 3 females) with different cultural backgrounds (Chinese, Iranian, Belgian, and Ethiopian), participated in the experiments. The average age of these observers was 28.0 years with a standard deviation of 3.5 years. For the green apple and the yellow lemon, another 10 observers (5 males and 5 females) with normal color vision participated in the experiments. They also came from different backgrounds, including Chinese, Indian, Belgian, Taiwanese, Ghanaian, Indonesian, Venezuelan and Albanian. Their average age was 26.0 years with a standard deviation of 3.1 years. Each session took around 30 minutes, during which 20 memory color matches (1 object × 10 starting points × 2 backgrounds) were completed. In total, 600 matches (3 objects × 10 starting points × 2 backgrounds × 10 observers) were made in the experiments.

3. Analysis

3.1 Observer variability

The inter-observer variability was evaluated per object and background illuminant by calculating the CIE 1976 u’₁₀v’₁₀ color difference, $\varDelta$(u′₁₀, v′₁₀), between the memory color of an individual observer and the mean memory color over all observers and then taking the mean (MCDM) [18]. Intra-observer variability was assessed by calculating the MCDM, with regard to each observer’s mean memory color, for the 10 matches associated with all 10 starting points and then averaging over each individual observer. The mean inter-observer MCDM values calculated for the yellow and EEW backgrounds were respectively 0.0143 and 0.0080 (grey cube), 0.0213 and 0.0246 (green apple), 0.0137 and 0.0112 (yellow lemon) in u’₁₀v’₁₀ units and the mean intra-observer MCDM values were respectively 0.0085 and 0.0062 (grey cube), 0.0096 and 0.0091 (green apple), 0.0112 and 0.0105 (yellow lemon) in u’₁₀v’₁₀ units. For the grey cube, both the repeatability and consistency were substantially higher for the EEW background than for the yellow background. A repeated measures ANOVA test shows that the impact of the background is significant for inter-observer variability (p = 0.004), and for intra-observer variability (p = 0.026). It shows that the inter-observer variability and the memory color region for the green apple are higher than for the grey cube and yellow lemon.

3.2 Adjustment track

For each individual observer, the tracks when adjusting the chromaticity from the starting point to the end point were recorded and are plotted in the CIE 1976 u’₁₀v’₁₀ chromaticity diagram. Among the observers, some had experience with MCM, while others were new to MCM. In the experiments with the grey cube, there were 5 naive observers and 5 experienced observers. For the green apple and yellow lemon experiments, there were 2 naive observers and 8 experienced observers. Figures 4(a) and 4(b) present the tracks for an experienced observer and Figs. 4(c) and 4(d) present the tracks for an inexperienced observer making an achromatic match (grey cube) under the yellowish and EEW background illumination, respectively. In Fig. 4, for each track, starting point (filled circle) and end point (open square) are represented in a color consistent with the color of the starting point in the experiment. Please note that the end points are relatively close to each other such that it might be difficult to observe them in Fig. 4. For both types of observer, as expected, the tracks deviate from a straight route to the achromatic match, as observers ‘search’ the chromaticity diagram for their final match. However, the tracks did not tend to cross the central region towards the complementary hue. In other words, if the observers found their match, they would either stop or explore that region a bit further in smaller steps, until they were satisfied that they have found their final match. Compared with experienced observers, naive observers tended to take more time and more steps to achieve a match. Their tracks also tended to be less direct covering a larger hue region. Note that some tracks were computed to be outside the spectral locus, because the recorded chromaticity was estimated from the projector calibration (which becomes less accurate close to its gamut borders) rather than from a direct measurement, as this would be too time-consuming. Similar results were found for other observers and other objects (green apple and yellow lemon).

 

Fig. 4. The recorded tracks for the achromatic matches with the grey cube from starting points (SP1 – SP10) to end points for two observers. In each track, the starting point is marked as a filled circle and the end point is marked as a square. (a) An experienced observer under the yellow illumination; (b) An experienced observer under EEW illumination; (c) An inexperienced observer under the yellow illumination; (d) An inexperienced observer under EEW illumination.

Download Full Size | PDF

The average steps and standard deviation in the matching track of 10 observers were plotted for each starting point and background chromaticity in Figs. 5(a)–5(c), which correspond to the grey cube, green apple and yellow lemon, respectively. Each time the observer pushed one of the arrow keys, one step was added to their total number of steps even though there was a pause and the subject decided to continue. Because the number of steps is not normally distributed, a generalized linear mixed model (GLMM), similar to a generalized linear model (GLM), but extended to non-normal data, was adopted to test the statistical significance of the group difference, with an α-level of 5%. The fixed-effects factors include the background chromaticity, starting point and observer type (experienced or naive observer) and the random-effects factor is the observer. The Chi-squared test of these fixed factors indicates that observer type (p < .0001), object (p < .0001), background (p < .0005) and starting points (p < .0001) all have significant impact on the number of steps. As the effect of object was significant, a GLMM analysis and the Chi-squared test were applied separately to the data subsets of the three objects. The results show that the background chromaticity has a significant impact on the number of steps for the green apple (p < .0005), but not for the grey cube (p > .05) and the yellow lemon (p > .05). Starting point had a significant impact on the number of steps for all the objects, including the yellow lemon (p < .0001), the grey cube (p < .0001) and the green apple (p < .0001). However, observer type is not a significant factor for all three objects (grey cube: p > .05; yellow lemon: p > .05; green apple: p > .05). It should be noted, however, that the statistical power related to only 2 naive and 8 experienced observers might have been too low to find a statistically significant effect.

 

Fig. 5. The average number of steps in a matching track for each of the 10 starting points and two background chromaticities. In each graph, the error bar represents the standard deviation of the 10 observers. (a) Grey Cube (b) Green Apple (c) Yellow lemon.

Download Full Size | PDF

3.3 The impact of starting points distribution

The end points (chromaticity coordinates for the memory color) for the grey cube averaged over all the observers for each starting point are shown in Figs. 6(a) and 6(b), corresponding to a yellow and an EEW background, respectively. Similar to Fig. 4, the color of each end point is represented by the color of the corresponding starting point. Firstly, it can be observed that end points for a yellowish starting point are always more yellowish than those with a blueish starting point and that end points with a green starting point are always more greenish than those with a red starting point. For the green apple and yellow lemon, similar results can be observed. The impact of starting point chromaticity was quantified by the correlation between the relative hue angles of the 8 chromatic starting points (SP1-4, SP6-9) and the relative hue angles of their corresponding end points in u’₁₀v’₁₀ space. Note that the relative hue direction for the starting point and end point is the vector from the average u’₁₀v’₁₀ chromaticity of 8 colorful starting points to each starting point, or the corresponding average end point to each individual end point, respectively. The Pearson correlation coefficients between the relative hue angle of the starting point and the end point for the yellow and EEW backgrounds are 0.97 and 0.95 (grey cube), 0.96 and 0.96 (green apple), 0.92 and 0.92 (yellow lemon). Starting point and end point were therefore highly correlated, indicating a substantial starting point bias and the need for a symmetrical distribution of the starting points around the expected chromaticity of the object’s memory color to balance out this bias.

 

Fig. 6. Distribution in the CIE 1976 u’₁₀v’₁₀ chromaticity diagram of the mean end points for all three familiar objects (mean over all 10 observers) for 10 starting points. (a) Yellow illumination (b) EEW illumination

Download Full Size | PDF

Therefore, the impact of a symmetrical distribution of the starting point chromaticity values on the accuracy of the end point was investigated more in depth. For each object, the average u’₁₀v’₁₀ chromaticity, obtained by averaging the 8 symmetrically distributed end points (SP1-4, SP6-9) of all 10 observers, has been used as ground truth. The color difference, DE_sp, between the average of n randomly selected starting points and the chromaticity of the neutral white-illuminated object has been used to characterize the symmetry of the randomly selected starting points around the center. The color difference, DE_ep, between the average of the corresponding n end points and the ground truth represents the accuracy of the end point (and thus the memory match). This was repeated for n varying from 1 to 10. The accuracy of the end point (DE_ep) was plotted against the symmetry (DE_sp) of the selected starting points in Fig. 7. The graphs in the three rows correspond to the grey cube, the yellow lemon and the green apple, respectively, and the graphs in the two columns correspond to the yellow and EEW illumination, respectively. The Pearson correlation coefficients between the symmetry of the starting points and the accuracy of the corresponding end points in terms of $\varDelta$(u′₁₀, v′₁₀) for the yellow and EEW background are 0.83 and 0.78 (grey cube), 0.72 and 0.88 (yellow lemon), 0.61 and 0.67 (green apple), respectively. For all three objects, the results confirm that the distribution of the starting point chromaticity has a substantial impact on the end point chromaticity and that more symmetric starting point distributions can give rise to a higher accuracy of the end point (memory match). What’s more, for yellow lemon and green apple, the impact was larger under EEW illumination than under yellow illumination.

 

Fig. 7. The relationship between the color difference between the average of n randomly selected starting points and the chromaticity of the EEW-illuminated object (DE_sp) and the accuracy of the end points (DE_ep). Three rows represent the different corresponding colors (grey cube, yellow lemon, green apple); columns 1 and 2 represent yellow and EEW illuminations respectively.

Download Full Size | PDF

3.4 The impact of starting points number

As more starting points (SP) are selected, it is expected that the accuracy of the results increases. However, more starting points result in longer experiments and the need to split them over several sessions to avoid observer fatigue. This trade-off is investigated with the aim of reducing experiment duration, while still retaining accurate and reproducible memory matches.

3.4.1 Precision and accuracy

The precision, related to reproducibility and repeatability, is the degree to which repeated measurements under unchanged conditions show the same results. The accuracy is the degree to which the measurement value approximates the true value. In the context of this experiment, precision and accuracy were evaluated for data subsets varying in number of starting points by selection from the full data set (10 observers, 8 starting points). Note that only the 8 colorful starting points (SP1-4, SP6-9) equally distributed along a hue circle were considered in this analysis. In the accuracy analysis, for each object and background, the ground truth was taken as the average u’₁₀v’₁₀ chromaticity, obtained by averaging all 8 end points of all 10 observers. For precision, the ground truth was taken as the mean u’₁₀v’₁₀ chromaticity, obtained by averaging all 8 end points but only for the selected observers. The precision and the accuracy were then calculated as the color difference ($\varDelta$(u′₁₀, v′₁₀)) in CIE 1976 u’₁₀v’₁₀ color space, between the mean chromaticity of the selected matching points, and their ground truth. Lower values indicate higher precision and accuracy. When all the 10 observers are selected, precision is identical to accuracy, because in this case they share the same ground truth.

3.4.2 Data processing

The precision and the accuracy of the experiment were analyzed in terms of the number of starting points (n_sp). Figure 8 schematically illustrates the data processing flow. The four steps in the procedure are explained below:

 

Fig. 8. Schematic diagram of data processing for investigating the impact of the number of starting points on precision and accuracy.

Download Full Size | PDF

In the first step, the experimentally determined matching data (end points), averaged over all the ten observers for each background and object, were resampled to obtain a large number of subsets with varying starting points number (from 1 to 8). Overall, 48 subsets were generated (8 starting points × 2 backgrounds × 3 objects). In a subset of n_sp starting points, there are $C_8^{{n_{sp}}}$ possible combinations, each composed of n_sp end points.

In the second step, within each subset, the average end point chromaticities for each of the $C_8^{{n_{sp}}}$ possible combinations was calculated.

Thirdly, for each subset the precision and the accuracy for all the possible end points were calculated as mentioned in section 3.4.1.

In the final step, for each of the 48 subsets, the maximum expected variation of precision and accuracy were estimated by the third quartiles of the dataset, which is taken to represent the possible lowest accuracy in end point chromaticity excluding extreme cases. The maximum variation of precision and accuracy as a function of starting point number (from 1 to 8) were obtained for the two backgrounds and the three objects.

3.4.3 Results

The maximum variation of accuracy, denoted as MaxDE_accuracy, was plotted as a function of starting point number for each of the different objects in Figs. 9(a) and 9(b), which correspond to the yellow and the EEW background, respectively. As precision is identical to accuracy in this context, only accuracy is discussed here. It is clear that, as expected, more starting points lead to a higher accuracy, i.e. lower MaxDE_accuracy value. The MaxDE_accuracy value is slightly lower under the EEW illuminant for the grey cube, which means higher accuracy, while higher accuracy is achieved under the yellow illuminant for the yellow lemon. The MaxDE_accuracy for 4 starting points for yellow and EEW illuminants are 0.0044 and 0.0038 (grey cube), 0.0029 and 0.0042 (yellow lemon), 0.0028 and 0.0027 (green apple), respectively; all values close to 0.0033, which corresponds to approximately one just noticeable difference (JND) or a three-step MacAdam ellipse [19], indicating that 4 starting points randomly selected from a super-set of uniformly distributed starting points can be considered adequate to provide accurate matching result.

 

Fig. 9. The impact of starting point numbers on MaxDE_accuracy value. In each graph, black, yellow and green curves represent grey cube, yellow lemon and green apple respectively. (a) Yellow illumination (b) EEW illumination

Download Full Size | PDF

As analyzed in Section 3.3, where all the possible starting points combination with different starting points number were taken into consideration, it has been demonstrated that a more symmetric distribution of the starting points around the expected final match point leads to a higher accuracy of the memory color chromaticity coordinates. In this section, the interaction between the distribution symmetry and the number of starting points was investigated. The level of symmetry of n_sp starting points was evaluated in terms of the normalized DE_sp, which is the u’₁₀v’₁₀ color difference between the average of the selected starting points and the chromaticity of the EEW-illuminated object normalized by the Root-Mean-Square of the semi-major and semi-minor axes of the ellipse fitted to the starting points distribution in the CIE u’₁₀v’₁₀ chromaticity diagram. Because of the different sizes of the starting points distribution ellipse for the three objects, the normalized DE_sp was adopted, instead of the absolute DE_sp value. Larger value corresponds to lower symmetry level. For the three objects, examples of the starting points distribution (n_sp= 3) at different symmetry levels, in terms of normalized DE_sp, are illustrated in Fig. 10.

 

Fig. 10. The examples of the starting points distribution (n_sp = 3) at different symmetry levels (5 columns) in terms of normalized DE_sp for the three objects including grey cube, yellow lemon and green apple (3 rows). The name of each column represents the symmetry range. For example, (0,0.2] means the symmetry value is larger than 0 but smaller than 0.2. The red cross in each ellipse is the chromaticity of the object illuminated by EEW, which is the symmetry center of full set of 8 starting points. The black cross is the average of the three selected starting points. The red, blue and black points represent three selected starting points.

Download Full Size | PDF

In the analysis, the accuracy and precision of the matching results were calculated for each starting points number and 8 different symmetry thresholds: 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, and infinity. At each threshold, only the starting point combinations with the normalized DE_sp lower than the threshold value were kept. Lower threshold values correspond to a more symmetric distribution of the starting points. So the infinity threshold indicates that none of the starting points combinations would be filtered from the group and the MaxDE_accuracy curve as a function of starting point number is identical to that in Fig. 9. In Fig. 11, the MaxDE_accuracy values obtained with the average matching results of 10 observers, have been plotted as a function of the number of starting points for the eight different thresholds of starting point symmetry. Figures 11(a) and 11(b) represent yellow and EEW illumination respectively, where the three subfigures represent the three objects (grey cube, yellow lemon and green apple). As shown in Fig. 11, larger threshold values lead to larger MaxDE_accuracy values, corresponding to lower accuracy, especially for the combination with a small number of starting points (n_sp= 2, 3, 4, 5). That this is especially apparent for smaller n_sp is partly due to the fact that for larger values the distribution of selected starting points become more symmetric by design: as n_sp grows, it becomes increasingly similar to the full set of 8 uniformly distributed starting points. The results in Fig. 11 again demonstrated that for different amounts of starting points ranging from 2 to 8, the starting points combinations with more symmetric distributions generate more accurate matches. Note that in Fig. 11 some data points for single starting points are missing, as these were filtered out due to their very low symmetry. Comparing the lowest threshold (0.2) with the highest threshold (infinity) for the grey cube, the MaxDE_accuracy values at n_sp= 3 are (0.0021 and 0.0061), and (0.0027 and 0.0048) for the yellow and EEW backgrounds, respectively. Similar results were found for the yellow lemon and green apple. The results indicate that the accuracy at the low threshold (highly symmetric starting point distributions) is nearly double (or more) that at the highest threshold.

 

Fig. 11. The MaxDE_accuracy value as a function of the number of starting points at 8 symmetry thresholds of the starting point distribution. The three subfigures represent the grey cube, yellow lemon and green apple respectively. (a) Yellow illumination (b) EEW illumination

Download Full Size | PDF

The MaxDE_accuracy determined for a symmetry threshold of 0.3 has been plotted in Fig. 12 as a function of the number of starting points for all three objects. A threshold value of 0.3 was chosen to ensure an adequate amount of starting points combinations for further analysis after threshold filtering. A comparison of the MaxDE_accuracy values plotted in Fig. 9 and Fig. 12, shows that the accuracy is substantially increased by selecting starting points that are more symmetrically distributed, especially at lower values of n_sp. When n_sp equals three, the MaxDE_accuracy value for yellow and EEW illuminants are 0.0032 and 0.0029 (grey cube), 0.0026 and 0.0033 (yellow lemon), 0.0023 and 0.0023 (green apple), respectively. These MaxDE_accuracy values for only 3 starting points are less than 0.0033 (corresponding to approximately a JND). Three symmetrically distributed starting points around the expected final match point are therefore adequate to provide accurate matching result.

 

Fig. 12. The impact of starting point numbers on MaxDE_accuracy value determined with a symmetry threshold value of 0.3. In each graph, black, yellow and green curves represent the grey cube, yellow lemon and green apple, respectively. (a) Yellow illumination (b) EEW illumination

Download Full Size | PDF

3.5 The impact of observer number

As this paper aims to provide guidelines for the memory color matching experiment, observer number, which is another factor that can influence the resulting accuracy of the matched memory color, should be taken into consideration. Involving more observers leads to a better estimate of the average ‘population’ observer, but will induce a higher experimental workload. The trade-off between precision and accuracy and the number of observers was investigated with the aim of reducing experiment duration, while still retaining accurate and stable (reproducible) results.

3.5.1 Data processing

In the experiment, the visual data from only 10 observers were collected. Therefore, to investigate the impact of observer number on accuracy for both smaller and larger groups of participants, a population of observers was simulated using the original data from the 10 participants and subsequently deploying the Gaussian process regression bootstrap resampling method [20]. This simulated population (used as ground truth) was then sampled to generate observer groups of different sizes. The various steps in the data processing flow are illustrated schematically in Fig. 13.

 

Fig. 13. Schematic diagram of the data processing work flow for investigating the impact of the observer number on accuracy.

Download Full Size | PDF

In the first step, for each object and each background, a bivariate normal probability distribution was fitted to the average (over all 8 matches corresponding to the 8 starting points) u’₁₀v’₁₀ chromaticity values of the 10 observers.

In the second step, 1000 data sets were randomly drawn from the simulated distribution for each background, object and observer number (n_ob), which ranged from 1 to 100. A total of 600 000 data sets (1000 repeats × 100 n_ob × 2 backgrounds × 3 objects) were generated.

In the third step, for each data set the average end point chromaticity over n_ob points was calculated.

In the fourth step, for each data set, the accuracy was calculated as the $\varDelta$(u′₁₀, v′₁₀) color difference between its average end point chromaticity and the ground truth (the center of the simulated population distribution).

In the fifth step, for each background, object and observer number, all 1000 average end point chromaticities were collected and the maximum variation of accuracy was estimated by the third quartiles.

3.5.2 Results

The MaxDE_accuracy has been plotted as a function of observer number for the three different objects under the yellow and EEW backgrounds in Figs. 14(a) and 14(b), respectively. As expected, as observer number increases, the accuracy increases. Green apple has the highest MaxDE_accuracy value and the lowest accuracy, especially for the EEW background, while the grey cube has the lowest MaxDE_accuracy value and the highest accuracy under both backgrounds. The MaxDE_accuracy value for a group of 10 observers for the yellow and EEW backgrounds are 0.0051 and 0.0022 (grey cube), 0.0055 and 0.0039 (yellow lemon), 0.0085 and 0.0090 (green apple), respectively. For the grey cube and the yellow lemon, 10 observers are therefore sufficient to provide accurate matching results, while for the green apple approximately 25 observers would be needed for the yellow illumination and about 55 for the EEW illumination to achieve a comparable accuracy.

 

Fig. 14. The impact of the number of observer number on the MaxDE_accuracy value. In each graph, red, blue and black curves represent grey cube, yellow lemon and green apple respectively. (a) Yellow illumination (b) EEW illumination.

Download Full Size | PDF

The large MaxDE_accuracy value of green apple may result from the large inter-observer variability. Figures 15(a) and 15(b) show the 95% confidence ellipses of the average matching results of 8 starting points for the three objects, under the yellow and EEW backgrounds, respectively. The size of the ellipse is a measure for the inter-observer variability for each object and a larger ellipse corresponds to larger variation among observers. From Figs. 15(a) and 15(b), it can be seen that the green apple is characterized by a substantially larger ellipse than the grey cube and the yellow lemon, especially under the EEW background. The size of the ellipses is consistent with the relative magnitudes of MaxDE_accuracy shown in Fig. 14. The large inter-observer variability in green apple can possibly be explained by the following two reasons. First, the large natural variation in “green” apple color, due to, among others, different types of green apples and varying stages of ripeness could have had an impact on the memory color between different observers. Secondly, the participation of observers from various cultural backgrounds in the memory color matching experiments may have led to an additional dispersion of the matching results for the green apple, as is for example reported in Smet et al. [21]. In summary, a green apple might be not very suitable for memory color matching.

 

Fig. 15. The fitted inter-observer variability ellipse of grey cube, yellow lemon and green apple at 95% confidence interval. (a) Yellow illumination (b) EEW illumination.

Download Full Size | PDF

4. Conclusion

Memory color matching and achromatic matching has been used in the past to study chromatic adaptation. These methods have several advantages over other more traditional asymmetric matching methods, but matches could be biased by the chromaticity of the starting point. To account for starting bias, starting points are typically counter-balanced. However, in a 2D chromaticity diagram it is less clear how many starting points are needed and how they are best distributed in color space. The effect of starting point chromaticity, number and distribution on matching precision and accuracy was therefore investigated with two backgrounds, a yellowish and a neutral white (EEW) background, and for three familiar objects, a grey cube, a green apple and a yellow lemon. The adapting field luminance was kept constant at approximately 600 cd/m², and the object luminance varied between 200 cd/m² (grey cube), 280 cd/ m² (green apple) and 330 cd/m² (yellow lemon). The effect of the number of observers was also included, with the aim of optimizing the efficiency of matching experiments by finding the minimum number of starting points and observers required to achieve accurate and reproducible matching results.

For each background chromaticity, 10 different starting points, including 8 colorful starting points (equally distributed along the hue circle centered at the EEW chromaticity), one at the EEW chromaticity and one starting point with the same chromaticity as the background are considered. During the experiment, the adjustment tracks of each observer have also been recorded.

Although the tracks from starting point to end point varied between observers, those of experienced observers were more direct than those of inexperienced observers. Tracks did not tend to cross over the central region towards the complementary hue.

An analysis of the end points distributions of average and individual observers showed that there was a substantial bias from the starting point chromaticity: memory color settings were biased towards the chromaticity of the starting point. It was also confirmed that more starting points resulted in a higher accuracy of the memory color match. Additionally, more symmetric starting point distributions around the chromaticity of the EEW-illuminated object gave rise to a higher accuracy of the average end point, especially under the EEW background chromaticity.

An analysis of the impact of the number of starting points on precision and accuracy of the match end points showed that 4 semi-symmetric starting points or 3 starting points with a symmetric distribution around the expected memory chromaticity were sufficient to obtain accurate and stable results. Compared to a full study with 10 starting points, this considerably reduces the time to complete the experiments.

An analysis of the impact of the number of observers on the accuracy of the match end points showed that 10 observers are sufficient to provide adequate accuracy for the grey cube and yellow lemon. However, for green apple, due to the larger inter-observer variability, more than 40 observers are required to obtain a comparable accuracy.