Check now Settings Resources Log in

Calculation of Metrics

This section describes in detail how the ImageZebra star ratings are computed. Note that the star ratings themselves do not indicate pass/fail. The user sets their own tolerance for individual metrics as what constitutes a passing rating.

Section 1 - General Metrics

Metric: Sampling Frequency

Description: Sampling Frequency is the number of pixels per inch as contained in the image file. It is often referred to as “resolution” in the image file header. The sampling frequency can be measured in both the horizontal and vertical directions, since directional differences may exist. This parameter informs us about the size of the original and also provides part of the data needed to determine the level of detail recorded in the file. ISO 12233:2014 defines the resolution measurements.

The measurement of the actual Sampling Frequency is computed by counting the number of pixels between two well defined marks on the original test pattern. Since the physical distance between these registration marks is known, the sampling frequency can be computed by dividing the number of pixels by the known distance in inches. The measured resolution for an imaging system can actually be different depending on the direction within the image. For the GoldenThread test pattern: The left and right dots are used to compute the actual x-resolution. The top and bottom dots are used to compute the actual y-resolution. Since the actual resolution is measured in 2 directions, only the minimum is used for comparison against the thresholds.

Sampling Frequency = Min(measured X Resolution, measured Y Resolution)

Metric: Reproduction Scale Accuracy

Description: Reproduction Scale Accuracy is a measurement of how well the actual sampling frequency matches that defined in the image file header. It is expressed as a percentage error. For example, if the image file header shows the resolution as 400 pixels/inch and the sampling frequency is measured as 396 ppi, the reproduction scale accuracy would have an error of -1.0% from the intended. (An error of -4 parts out of 400 = -1.0%)

The measurement of the actual Sampling Frequency is computed by counting the number of pixels between two well defined marks on the original test pattern. Since the physical distance between these registration marks is known, the sampling frequency can be computed by dividing the number of pixels by the known distance in inches. For the GoldenThread test pattern: The left and right dots are used to compute the actual x-resolution. The top and bottom dots are used to compute the actual y-resolution.

Reproduction Scale Accuracy is computed by first finding the actual x-Resolution and actual y-Resolution, and then computing xMag and yMag by taking computed resolution / indicated resolution, where the indicated resolution is retrieved from the image header. The error is computed as the percentage difference.

xMag and yMag - the magnification from nominal. 1.0 is nominal. A value < 1.0 means the image is smaller than nominal. A number very different (such as something like 0.5) may mean the resolution recorded in the image header does not match the actual resolution.

The magnification is computed in both dimensions, and the largest deviation from 1.0 is used to determine a single magnification. Details are as follows:

xMag = xRes / xResHeader;
yMag = yRes / yResHeader;

if (ABS(xMag-1.0) > ABS(yMag-1.0))
        Magnification = xMag;
else
        Magnification = yMag;

Section 2 - Spatial Frequency Response (SFR) Metrics

Description: The spatial frequency response (SFR) is a measure of the contrast gradient across an edge. It is related to the sharpness, focus, and ability to resolve fine details. The SFR is a functional relationship, which defines the system’s ability to maintain contrast of increasing smaller details. There are several single valued metrics to describe the SFR function. These include measurements at specific points on the curve, such as the 50% and 10% modulation points as well as the response at the highest frequency supported by the sampling frequency. ISO 12233:2000, ISO 16067-1:2003, and ISO 16067-2:2004 define the MTF/SFR measurement. (Note: sometimes in this documentation the SFR is also referred to as the Modulation Transfer Function, MTF. The SFR is an easily computed surrogate for MTF, however, in the purest sense SFR may not give the same results due to assumptions about linearity and the response at the zero frequency.) These metrics provide a numerical value to describe the visual effects seen in the images below.

image-20210916090251808 image-20210916090302025

The SFR/MTF metrics are computed by looking at the slanted edges of the image. For the GoldenThread target, these are located as follows:

The edges are grouped into two groups:

Each edge is processed through the SFR edge analysis code available from Burns Digital Imaging LLC (BurnsDigitalImaging.com ) The code reports SFR information based on frequency as shown in the example below:

The code computes MTF_10 and MTF_50 for 4 planes: R, G, B, and L*. We are only interested in L* The following data is extracted:

On the Golden Thread test pattern, there are 5 slightly rotated squares, which provide the slanted edges for the SFR analysis. Each square has 2 horizontal and 2 vertical edges. For this SFR analysis, the pairs of horizontal and vertical results are averaged to produce 5 horizontal and 5 vertical values for each metric.

For example:

Similarly for the C, D, and E patterns.

A similar set is constructed for MTF_50 measurements. In this way each image has a total of 10 MTF_10 values, and 10 MTF_50 values.

Metric SFR10 (Sampling Efficiency)

Determine the frequency (in cy/pix) where the SFR has a response value of 10%. Then divide this frequency by 0.5 (the half sampling / Nyquist Frequency)

Come in the Y axis and see what sampling frequency has a SFR response of 0.10 (10%). Then divide by nyquist.

Sample value: The plot above shows a MTF10 Sampling Efficiency of about 0.35 / 0.5 = 0.70 or 70%

This results in 10 ratings, one for each edge.

Metric: SFR10 (Response at Half Sampling Frequency)

This is actually the SFR response at the Nyquist Freq. (0.5 cy/pixel).

Come in the X-axis at half the sampling resolution, and see what the Y value is.

Use the SFR graph and enter the X-axis at a value of 0.5 cy/pix. Find the value of the SFR response at 0.5 cy/pix for the L* channel only.

Sample value: The plot above shows a MTF10 SFR Response of about 0.05

This results in 10 ratings, one for each edge.

Metric: SFR50 (50% SFR)

Find the frequency where the modulation is 50%.

Then divide this frequency by the half sampling frequency, (which if we are in cycles/pixel, mathematically is the same as multiplying by two, since the half sampling frequency is 0.5). This gives a number like 0.45.

Multiply this by 100 to get something like 45

Sample value: The plot above shows:

This results in 10 ratings, one for each edge.

Note that when computing the star rating, these values must satisfy both a lower and upper bound.

Metric: Sharpening

Computed by finding the Max. MTF of L*

Sample value: The plot above shows a Max MTF of 1.0

Section 3 - Colorimetric Metrics

The colorimetric metrics are computed from looking at individual patches. The data collected from the patches is done from two areas: the Region of Interest, and the Sample Window

The Region of Interest (ROI) is a slight subset of the entire patch. This is typically set to 80% of the ideal patch (to account for the registration being not perfect)

The Sample Window is found by searching the region of interest (ROI) for a window that has the lowest noise. Parameters used to control this are:

The GoldenThread.jpg image was checked for patch size. A single Ideal Patch is 153 x 134 pixels. So, make it a tiny bit smaller: 150 x 130 pixels. The “Corner” patches are a bit bigger - 158 x 158 pixels.

Metric: Noise (L*) (Colorimetric mode)

This uses L*.

This is very similar to the Noise (Digital Counts) metric, but this metric uses L* instead of the digital counts.

The Noise does NOT use the sample window of the patch, but instead uses the entire Region of Interest of the greyscale patches. The Noise is computed by converting every individual pixel from RGB to LAB (as done in steps 2 thru 4 above) and then calculating the standard deviation of all L values. This is repeated for all 12 greyscale patches, and results in 12 unique individual ratings, where each individual rating is the standard deviation of the L values of a given patch.

The summary rating is determined by finding the LOWEST rating of all individual ratings.

Metric: Noise (Digital Counts mode)

This is very similar to the Noise (L) metric, but this metric uses the Digital Counts instead of L

The Noise does NOT use the sample window, but uses the entire Region of Interest of the greyscale patches. The Noise is computed by calculating the standard deviation of all color components (R, G, and B values.) Noise (color) = this standard deviation of all (color) pixels. This is repeated for all 12 greyscale patches, and results in 12 unique individual ratings, where each individual rating is the standard deviation of the L* values of a given patch.

The patch score is computed by fining the minimum of these:

? Patch_Score = Min(Noise(R), Noise(G), Noise(B))

The summary rating is determined by finding the LOWEST rating of all individual ratings.

Metric: Tone Response (OECF) (Colorimetric mode)

The MeasuredLAB of each Greyscale patch is computed and compared against the IdealLAB. This results in a dE value for each patch, for a total of 12 individual data points.

The summary rating is determined by finding the star rating of the LOWEST individual rating.

Metric: Tone Response (OECF) (Digital Counts mode)

The AverageRGB of each of the 12 Greyscale patchs is computed. The individual components are: AverageR, AverageG, and AverageB. The IdealRGB of the patch is known.

For each patch compute the per-channel differences: Delta_R = Abs(AverageR - IdealR) Delta_G = Abs(AverageG - IdealG) Delta_B = Abs(AverageB - IdealB)

The patch score is computed by averaging these: Patch_Score = Average(Delta_R, Delta_G, Delta_B)

The summary rating is determined by finding the LOWEST rating of all individual ratings.

Metric: White Balance (Colorimetric mode)

For each of the greyscale patches, the MeasuredLAB of each patch is computed and compared against the IdealLAB. This results in a dAB value for each patch.

Metric: White Balance (Digital Counts mode)

For each of the greyscale patches, using AverageRGB (which is an RGB value computed by averaging all the R values, G, values, and B values in the sample window), compute the difference in digital counts between the R, G, and B channels. We expect the neutral patches to have equal R, G, and B values. This metric measures the error or Non-neutrality.

So, let MaxDiff = Max (Abs(R-G), Abs(G-B), Abs( R-B) )

This value is compared against the star rating thresholds.

The summary rating is determined by finding the LOWEST rating of all individual ratings.

Metric: ColorAccuracy (Mean dE2000)

Metric: Color Accuracy (90th Percentile dE2000)

The MeasuredLAB of all patches (color and neutral) are computed and compared against the IdealLAB. This results in a dE2000 value for each patch.

The 90th percentile is determined by finding the smallest dE2000 value that is greater than or equal to 90% of the dE2000 patch values.

Metric: Lightness Nonuniformity

This metric only applies to Colorimetric mode. (There is no corresponding metric in Digital Counts mode)

The MeasuredLAB of each of the 20 corner patch is computed. This results in 20 LAB values. Then the standard deviation (Stdev) and Mean of all of the 20 L* values is are computed. Next, compute the Coefficient of Variation and express it as a percentage:

? Coefficient of Variation (%) = ( Stdev/Mean ) * 100

Calculation of dE and dAB:

Calculating dE

The algorithm used to compute dE can be found here:

http://www.brucelindbloom.com/Eqn_DeltaE_CIE2000.html

and here:

https://en.wikipedia.org/wiki/Color_difference#CIEDE2000

There s a handy on-line tool to check particular values here:

http://www.brucelindbloom.com/ColorDifferenceCalc.html

Calculation of da*b*:

The calculation of da*b* is very similar, it just does not use the first term.

From the http://www.brucelindbloom.com/Eqn_DeltaE_CIE2000.html, the calculation of dE is as follows. If you cross out the first term, it becomes the calculation of da b:

Glossary of Raw Data Terms

PixelValue - the average RGB value for the Region of Interest

(For all pixels in the Region of Interest, average all R, G, and B)

XyzD65 - Takes PixelValue from above, and converts it to XYZ as follows:

First use the color profile to convert it to XYZ. But this assumes D50.

So then compute:

for i = 0; i < 3; i++
    xyzD65[i] = xyz[i] * xyzD65White[i] / xyzD50White[i]

where:

    xyzD50White = 96.42, 100.0, 82.49;
    xyzD65White = 95.04, 100.0, 108.88;

LAB - Convert xyzD65 to LAB using the equations Bruce Lindbloom gives here:

http://www.brucelindbloom.com/Eqn_XYZ_to_Lab.html

Note that at the bottom of the page he provides different values for constants e and k, stating one value is "Actual CIE standard" and the other is "Intent of the CIE standard". The values by ImageZebra are the "Intent of the CIE standard". The white point used is D65.

Ideal_LAB- This is the ideal LAB value, either Encoded in the 2-D barcode, or known for a given target.

Ideal_RGB- This is the Ideal_LAB value converted to RGB as follows:

- If an embedded profile exists, use it to convert Ideal_LAB to RGB using D50 white point

- If an embedded profile does NOT exist, use AdobeRGB1998 using D65 as the white point!!

Right now it is not clear why the choice was made to use D50 for one, and D65 for the other!! Likely a bug, which needs to be investigated.