Robert Charles Carter, Pedro A. García, Manuel Melgosa, and Michael H. Brill, "Metrics of color-difference formula improvement," J. Opt. Soc. Am. A 39, 1360-1370 (2022)
Metrics of color-difference formula improvement (i.e., standardized residual sum of squares and Pearson product moment correlation) are shown to convey the same information. Furthermore, each metric has two computational forms that assume different linear data models, specifically, with or without an ordinate intercept. It is essential to choose a computational form that matches the data model. We recommend explicitly declaring whether or not the data have been centered, i.e., by subtracting the mean value from each datum, to match the intercept-free data model. Statistical testing of the metrics assumes independent, normally distributed randomness of residuals from the data model, and homogeneous variance. Procedures consistent with these assumptions include robust statistical tests, homogenizing data transformations, and meta-analysis.
Data underlying the results presented in this paper are available in Dataset 1, Ref. [15] and may be obtained also from Manuel Melgosa upon reasonable request. See Code 1 [21] for supporting content, providing code that generated results in Tables 2 and 3.
21. P. A. Garcia, “R code for ordinate intercept confidence limits (Table 2) and for calculating Hittner et al. [20] statistical significance test of non-independent correlations (Table 3, Fig. 3),” figshare, 2021, https://doi.org/10.6084/m9.figshare.20151461.
Cited By
You do not have subscription access to this journal. Cited by links are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.
You do not have subscription access to this journal. Figure files are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.
You do not have subscription access to this journal. Article tables are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.
You do not have subscription access to this journal. Equations are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.
Effect of Centered or Uncentered Data on an Identity Involving Squares of STRESS or and or , Based on CIEDE2000 as , and in the CIE Combined Corrected Dataseta
See Appendix A for information about the CIE combined corrected dataset.
Centered data have had their means (i.e., of $\Delta E$ and $\Delta V$) subtracted from $\Delta {E_i}$ and $\Delta {V_i}$, respectively, for $i = 1, 2, {\ldots }n$, before calculating STRESS, $\textit{STRESS}_2$, or PPMC.
Table 2.
PPMC, , between (13 Color-Difference Formulas) and with Their Associated Slopes (), Ordinate Intercepts (), and Average Color Differences, Calculated from the CIE Combined Corrected Dataset
Color-Difference Formulas ()
Average Values
Intercept in
Lower 95% Confidence Bound of
Upper 95% Confidence Bound of
Slope in
CIELAB
0.717
3.97
0.543
0.542
0.543
1.468
CIELUV
0.683
4.94
0.573
0.573
0.573
1.730
CMC
0.840
2.87
0.261
0.260
0.261
1.036
BFD
0.828
3.51
0.291
0.291
0.291
1.271
CIE94
0.825
2.54
0.284
0.284
0.284
0.908
CIEDE2000
0.871
2.45
0.214
0.214
0.214
0.906
DIN99d
0.850
2.88
0.220
0.220
0.220
1.024
CAM02-SCD
0.858
2.42
0.199
0.199
0.199
0.858
CAM02-UCS
0.852
2.87
0.210
0.209
0.210
1.013
OSA-GP
0.844
2.40
0.224
0.223
0.224
0.841
OSA-GP_Eu
0.848
2.34
0.219
0.219
0.219
0.825
ULAB
0.770
2.52
0.245
0.245
0.245
0.752
Wlab
0.831
2.87
0.289
0.289
0.289
1.045
Table 3.
Results for a Statistical Test Procedure, Using Fisher’s , Recommended by Hittner et al. [20,21] for the Difference of Non-Independent PPMCsa
CIELUV
CMC
BFD
CIE94
CIE00
DIN99d
CAM02-SCD
CAM02-UCS
OSA-GP
OSA-GP-Eu
ULAB
Wlab
CIELAB
13.9
39.3
34.3
33.3
49.6
40.6
43.0
45.6
37.6
38.6
12.5
34.0
CIELUV
–
42.5
38.7
36.4
52.1
43.8
48.2
49.9
45.8
46.2
18.3
40.7
CMC
–
7.30
8.52
21.3
7.36
11.6
7.56
1.97
5.06
23.0
4.39
BFD
–
1.08
23.2
13.3
16.1
12.2
9.17
11.4
17.9
1.61
CIE94
–
31.7
19.0
19.7
19.0
8.23
11.3
19.1
3.95
CIE00
–
18.7
10.7
15.4
16.6
15.1
38.4
22.2
DIN99d
–
7.47
1.39
4.45
1.60
28.9
11.0
CAM02-SCD
–
9.48
9.87
7.20
30.6
14.7
CAM02-UCS
–
5.13
2.53
28.9
12.1
OSA-GP
–
10.3
23.2
6.07
OSA-GP-Eu
–
25.5
8.73
ULAB
–
20.2
Cells representing those $\Delta E$ pairs lacking significance at 0.05 level are underlined.
Table 4.
Effects of (Variance-Homogenizing) Power Transformations and Meta-Analysis to Normalize the Distribution of Residuals Based on the CIE Combined Corrected Dataseta
Combined Data versus Meta-Analysis
Transformed versus Raw Data
MS Excel Skew
MS Excel Kurt
Combined corrected dataset (weighted: 11,273 color pairs)
Raw
2.92
14.59
−0.35
2.60
Unweighted combined corrected dataset (UDS of 3,813 color pairs)
Raw
2.11
16.88
1.88
16.71
Meta-analysis of UDS: BFD, Leeds, RIT-DuPont, and Witt (3,813 color pairs)
Raw
1.16
3.50
0.09
2.97
Meta-analysis of UDS: BFD (three sub-studies), Leeds (two sub-studies), RIT-DuPont, Witt (3,813 color pairs)
Raw
0.84
2.83
0.56
3.20
Microsoft (MS) Excel skew and kurt would be zero for a normal distribution. Tabulated meta-analysis skew and kurt are the weighted average of statistics in the individual studies, where the weights are the inverses of the variances of residuals from each study’s linear data model. UDS signifies the unweighted dataset (${{n}} = {{3{,}813}}$), a subset of the combined corrected dataset having no redundant data.
Table 5.
Results of Simulation (Where “Fit” Means H (Kolmogorov–Smirnov) )
0.3
0.4
0.5
0.6
0.7
0.8
0.9
10
Fit
Fit
No fit
No fit
No fit
No fit
No fit
12
Fit
Fit
Fit
Fit
No fit
Fit
No fit
15
Fit
Fit
Fit
Fit
Fit
Fit
Fit
20
Fit
Fit
Fit
Fit
Fit
Fit
Fit
30
Fit
Fit
Fit
Fit
Fit
Fit
Fit
50
Fit
Fit
Fit
Fit
Fit
Fit
Fit
Tables (5)
Table 1.
Effect of Centered or Uncentered Data on an Identity Involving Squares of STRESS or and or , Based on CIEDE2000 as , and in the CIE Combined Corrected Dataseta
See Appendix A for information about the CIE combined corrected dataset.
Centered data have had their means (i.e., of $\Delta E$ and $\Delta V$) subtracted from $\Delta {E_i}$ and $\Delta {V_i}$, respectively, for $i = 1, 2, {\ldots }n$, before calculating STRESS, $\textit{STRESS}_2$, or PPMC.
Table 2.
PPMC, , between (13 Color-Difference Formulas) and with Their Associated Slopes (), Ordinate Intercepts (), and Average Color Differences, Calculated from the CIE Combined Corrected Dataset
Color-Difference Formulas ()
Average Values
Intercept in
Lower 95% Confidence Bound of
Upper 95% Confidence Bound of
Slope in
CIELAB
0.717
3.97
0.543
0.542
0.543
1.468
CIELUV
0.683
4.94
0.573
0.573
0.573
1.730
CMC
0.840
2.87
0.261
0.260
0.261
1.036
BFD
0.828
3.51
0.291
0.291
0.291
1.271
CIE94
0.825
2.54
0.284
0.284
0.284
0.908
CIEDE2000
0.871
2.45
0.214
0.214
0.214
0.906
DIN99d
0.850
2.88
0.220
0.220
0.220
1.024
CAM02-SCD
0.858
2.42
0.199
0.199
0.199
0.858
CAM02-UCS
0.852
2.87
0.210
0.209
0.210
1.013
OSA-GP
0.844
2.40
0.224
0.223
0.224
0.841
OSA-GP_Eu
0.848
2.34
0.219
0.219
0.219
0.825
ULAB
0.770
2.52
0.245
0.245
0.245
0.752
Wlab
0.831
2.87
0.289
0.289
0.289
1.045
Table 3.
Results for a Statistical Test Procedure, Using Fisher’s , Recommended by Hittner et al. [20,21] for the Difference of Non-Independent PPMCsa
CIELUV
CMC
BFD
CIE94
CIE00
DIN99d
CAM02-SCD
CAM02-UCS
OSA-GP
OSA-GP-Eu
ULAB
Wlab
CIELAB
13.9
39.3
34.3
33.3
49.6
40.6
43.0
45.6
37.6
38.6
12.5
34.0
CIELUV
–
42.5
38.7
36.4
52.1
43.8
48.2
49.9
45.8
46.2
18.3
40.7
CMC
–
7.30
8.52
21.3
7.36
11.6
7.56
1.97
5.06
23.0
4.39
BFD
–
1.08
23.2
13.3
16.1
12.2
9.17
11.4
17.9
1.61
CIE94
–
31.7
19.0
19.7
19.0
8.23
11.3
19.1
3.95
CIE00
–
18.7
10.7
15.4
16.6
15.1
38.4
22.2
DIN99d
–
7.47
1.39
4.45
1.60
28.9
11.0
CAM02-SCD
–
9.48
9.87
7.20
30.6
14.7
CAM02-UCS
–
5.13
2.53
28.9
12.1
OSA-GP
–
10.3
23.2
6.07
OSA-GP-Eu
–
25.5
8.73
ULAB
–
20.2
Cells representing those $\Delta E$ pairs lacking significance at 0.05 level are underlined.
Table 4.
Effects of (Variance-Homogenizing) Power Transformations and Meta-Analysis to Normalize the Distribution of Residuals Based on the CIE Combined Corrected Dataseta
Combined Data versus Meta-Analysis
Transformed versus Raw Data
MS Excel Skew
MS Excel Kurt
Combined corrected dataset (weighted: 11,273 color pairs)
Raw
2.92
14.59
−0.35
2.60
Unweighted combined corrected dataset (UDS of 3,813 color pairs)
Raw
2.11
16.88
1.88
16.71
Meta-analysis of UDS: BFD, Leeds, RIT-DuPont, and Witt (3,813 color pairs)
Raw
1.16
3.50
0.09
2.97
Meta-analysis of UDS: BFD (three sub-studies), Leeds (two sub-studies), RIT-DuPont, Witt (3,813 color pairs)
Raw
0.84
2.83
0.56
3.20
Microsoft (MS) Excel skew and kurt would be zero for a normal distribution. Tabulated meta-analysis skew and kurt are the weighted average of statistics in the individual studies, where the weights are the inverses of the variances of residuals from each study’s linear data model. UDS signifies the unweighted dataset (${{n}} = {{3{,}813}}$), a subset of the combined corrected dataset having no redundant data.
Table 5.
Results of Simulation (Where “Fit” Means H (Kolmogorov–Smirnov) )
Add to BibSonomy