Last Updated: 09/16/2021

[TOC]

Calculation of Metrics

This section describes in detail how the ImageZebra star ratings are computed. Note that the star ratings themselves do not indicate pass/fail. The user sets their own tolerance for individual metrics as what constitutes a passing rating.

Section 1 - General Metrics

Metric: Sampling Frequency

Description: Sampling Frequency is the number of pixels per inch as contained in the image file. It is often referred to as “resolution” in the image file header. The sampling frequency can be measured in both the horizontal and vertical directions, since directional differences may exist. This parameter informs us about the size of the original and also provides part of the data needed to determine the level of detail recorded in the file. ISO 12233:2014 defines the resolution measurements.

Image Zebra uses the image header information to define the intended Sampling Frequency. The FADGI star rating is based on this value extracted from the image header. The actual measured Sampling Frequency is used to compute the Reproduction Scale Accuracy (see below). xResolution and yResolution are often the same in the image file header. ImageZebra uses the minimum of these, so:

Sampling Frequency = Min(X Resolution from Header, Y Resolution from Header)

The star rating for Specification: FADGI Documents (Unbound): General Collections is as follows:

Metric: Reproduction Scale Accuracy

Description: Reproduction Scale Accuracy is a measurement of how well the actual sampling frequency matches that defined in the image file header. It is expressed as a percentage error. For example, if the image file header shows the resolution as 400 pixels/inch and the sampling frequency is measured as 396 ppi, the reproduction scale accuracy would have an error of -1.0% from the intended. (An error of -4 parts out of 400 = -1.0%)

The measurement of the actual Sampling Frequency is computed by counting the number of pixels between two well defined marks on the original test pattern. Since the physical distance between these registration marks is known, the sampling frequency can be computed by dividing the number of pixels by the known distance in inches. For the GoldenThread test pattern: The left and right dots are used to compute the actual x-resolution. The top and bottom dots are used to compute the actual y-resolution.

Reproduction Scale Accuracy is computed by first finding the actual x-Resolution and actual y-Resolution, and then computing xMag and yMag by taking computed resolution / indicated resolution, where the indicated resolution is retrieved from the image header. The error is computed as the percentage difference.

xMag and yMag - the magnification from nominal. 1.0 is nominal. A value < 1.0 means the image is smaller than nominal. A number very different (such as something like 0.5) may mean the resolution recorded in the image header does not match the actual resolution.

The magnification is computed in both dimensions, and the largest deviation from 1.0 is used to determine a single magnification. Details are as follows:

xMag = xRes / xResHeader;
yMag = yRes / yResHeader;

if (ABS(xMag-1.0) > ABS(yMag-1.0))
        Magnification = xMag;
else
        Magnification = yMag;

The star rating for Specification: FADGI Documents (Unbound): General Collections is as follows:

Section 2 - Spatial Frequency Response (SFR) Metrics

Description: The spatial frequency response (SFR) is a measure of the contrast gradient across an edge. It is related to the sharpness, focus, and ability to resolve fine details. The SFR is a functional relationship, which defines the system’s ability to maintain contrast of increasing smaller details. There are several single valued metrics to describe the SFR function. These include measurements at specific points on the curve, such as the 50% and 10% modulation points as well as the response at the highest frequency supported by the sampling frequency. ISO 12233:2000, ISO 16067-1:2003, and ISO 16067-2:2004 define the MTF/SFR measurement. (Note: sometimes in this documentation the SFR is also referred to as the Modulation Transfer Function, MTF. The SFR is an easily computed surrogate for MTF, however, in the purest sense SFR may not give the same results due to assumptions about linearity and the response at the zero frequency.) These metrics provide a numerical value to describe the visual effects seen in the images below.

image-20210916090251808 image-20210916090302025

The SFR/MTF metrics are computed by looking at the slanted edges of the image. For the GoldenThread target, these are located as follows:

The edges are grouped into two groups:

Each edge is processed through the SFR edge analysis code available from Burns Digital Imaging LLC (BurnsDigitalImaging.com ) The code reports SFR information based on frequency as shown in the example below:

The code computes MTF_10 and MTF_50 for 4 planes: R, G, B, and L*. We are only interested in L* The following data is extracted:

On the Golden Thread test pattern, there are 5 slightly rotated squares, which provide the slanted edges for the SFR analysis. Each square has 2 horizontal and 2 vertical edges. For this SFR analysis, the pairs of horizontal and vertical results are averaged to produce 5 horizontal and 5 vertical values for each metric.

For example:

Similarly for the C, D, and E patterns.

A similar set is constructed for MTF_50 measurements. In this way each image has a total of 10 MTF_10 values, and 10 MTF_50 values.

Metric SFR10 (Sampling Efficiency)

Determine the frequency (in cy/pix) where the SFR has a response value of 10%. Then divide this frequency by 0.5 (the half sampling / Nyquist Frequency)

Come in the Y axis and see what sampling frequency has a SFR response of 0.10 (10%). Then divide by nyquist.

Sample value: The plot above shows a MTF10 Sampling Efficiency of about 0.35 / 0.5 = 0.70 or 70%

This results in 10 ratings, one for each edge.

The star rating for Specification: FADGI Documents (Unbound): General Collections is as follows:

Metric: SFR10 (Response at Half Sampling Frequency)

This is actually the SFR response at the Nyquist Freq. (0.5 cy/pixel).

Come in the X-axis at half the sampling resolution, and see what the Y value is.

Use the SFR graph and enter the X-axis at a value of 0.5 cy/pix. Find the value of the SFR response at 0.5 cy/pix for the L* channel only.

Sample value: The plot above shows a MTF10 SFR Response of about 0.05

This results in 10 ratings, one for each edge.

The star rating for Specification: FADGI Documents (Unbound): General Collections is as follows:

Metric: SFR50 (50% SFR)

Find the frequency where the modulation is 50%.

Then divide this frequency by the half sampling frequency, (which if we are in cycles/pixel, mathematically is the same as multiplying by two, since the half sampling frequency is 0.5). This gives a number like 0.45.

Multiply this by 100 to get something like 45

Sample value: The plot above shows:

This results in 10 ratings, one for each edge.

The star rating for Specification: FADGI Documents (Unbound): General Collections is as follows:

Metric: Sharpening

Computed by finding the Max. MTF of L*

Sample value: The plot above shows a Max MTF of 1.0

The star rating for Specification: FADGI Documents (Unbound): General Collections is as follows

Section 3 - Colorimetric Metrics

The colorimetric metrics are computed from looking at individual patches. The data collected from the patches is done from two areas: the Region of Interest, and the Sample Window

The Region of Interest (ROI) is a slight subset of the entire patch. This is typically set to 80% of the ideal patch (to account for the registration being not perfect)

The Sample Window is found by searching the region of interest (ROI) for a window that has the lowest noise. Parameters used to control this are:

The GoldenThread.jpg image was checked for patch size. A single Ideal Patch is 153 x 134 pixels. So, make it a tiny bit smaller: 150 x 130 pixels. The “Corner” patches are a bit bigger - 158 x 158 pixels.

Definitions: AverageRGB, MeasuredLAB, DeltaE and DeltaAB:

The computation of the various metrics use AverageRGB, MeasuredLAB, DeltaE and DeltaAB.

Here is how these values are computed:

  1. The R, G, and B values for all pixels in the Sample Window are averaged, resulting in a single AverageRGB. This has 3 components: R, G, and B where each component is simply the averages of that component in the Sample Window.
  2. The AverageRGB is converted to XYZ_D50. This is done with the following: viewing illuminance: 200 lux viewing white point: CIE D50 media white point: "that of a perfectly reflecting diffuser" -- D50 media black point: 0 lux or 0 Reflectance flare: 1 percent surround: 20percent of the media white point
  3. The XYZ_D50 value is then converted to XYZ_D65 by using, for each of the 3 components: XYZ_D65 = XYZ_D50 * (xyzD65White / xyzD50White ) Where the white points used are: xyzD50White = 96.42, 100.0, 82.51 xyzD65White = 95.04, 100.0, 108.88
  4. Convert the XYZ_D65 to LAB, using the D65 illuminant. This is the MeasuredLAB
  5. The DeltaE for the patch is computed as: dE = the dE2000 of this LAB. (See Calculating dE below)
  6. The Deltaa b for the patch is computed just like DeltaE, except without using the L component. (See Calculating da b* below)

Metric: Noise (L*) (Colorimetric mode)

This uses L*.

This is very similar to the Noise (Digital Counts) metric, but this metric uses L* instead of the digital counts.

The Noise does NOT use the sample window of the patch, but instead uses the entire Region of Interest of the greyscale patches. The Noise is computed by converting every individual pixel from RGB to LAB (as done in steps 2 thru 4 above) and then calculating the standard deviation of all L values. This is repeated for all 12 greyscale patches, and results in 12 unique individual ratings, where each individual rating is the standard deviation of the L values of a given patch.

The star rating for Specification: FADGI Documents (Unbound): General Collections is as follows:

The summary rating is determined by finding the LOWEST rating of all individual ratings.

Metric: Noise (Digital Counts mode)

This is very similar to the Noise (L) metric, but this metric uses the Digital Counts instead of L

The Noise does NOT use the sample window, but uses the entire Region of Interest of the greyscale patches. The Noise is computed by calculating the standard deviation of all color components (R, G, and B values.) Noise (color) = this standard deviation of all (color) pixels. This is repeated for all 12 greyscale patches, and results in 12 unique individual ratings, where each individual rating is the standard deviation of the L* values of a given patch.

The patch score is computed by fining the minimum of these:

​ Patch_Score = Min(Noise(R), Noise(G), Noise(B))

The star rating for Specification: FADGI Documents (Unbound): General Collections is as follows:

The summary rating is determined by finding the LOWEST rating of all individual ratings.

Metric: Tone Response (OECF) (Colorimetric mode)

The MeasuredLAB of each Greyscale patch is computed and compared against the IdealLAB. This results in a dE value for each patch, for a total of 12 individual data points.

The star rating for Specification: FADGI Documents (Unbound): General Collections is as follows:

The summary rating is determined by finding the star rating of the LOWEST individual rating.

Metric: Tone Response (OECF) (Digital Counts mode)

The AverageRGB of each of the 12 Greyscale patchs is computed. The individual components are: AverageR, AverageG, and AverageB. The IdealRGB of the patch is known.

For each patch compute the per-channel differences: Delta_R = Abs(AverageR - IdealR) Delta_G = Abs(AverageG - IdealG) Delta_B = Abs(AverageB - IdealB)

The patch score is computed by averaging these: Patch_Score = Average(Delta_R, Delta_G, Delta_B)

The star rating for Specification: FADGI Documents (Unbound): General Collections is as follows:

The summary rating is determined by finding the LOWEST rating of all individual ratings.

Metric: White Balance (Colorimetric mode)

For each of the greyscale patches, the MeasuredLAB of each patch is computed and compared against the IdealLAB. This results in a dAB value for each patch.

The star rating for Specification: FADGI Documents (Unbound): General Collections is as follows:

Metric: White Balance (Digital Counts mode)

For each of the greyscale patches, using AverageRGB (which is an RGB value computed by averaging all the R values, G, values, and B values in the sample window), compute the difference in digital counts between the R, G, and B channels. We expect the neutral patches to have equal R, G, and B values. This metric measures the error or Non-neutrality.

So, let MaxDiff = Max (Abs(R-G), Abs(G-B), Abs( R-B) )

The star rating for Specification: FADGI Documents (Unbound): General Collections is as follows:

The summary rating is determined by finding the LOWEST rating of all individual ratings.

Metric: Color Accuracy (dE2000)

This metric only applies to Colorimetric mode. (There is no corresponding metric in Digital Counts mode)

The MeasuredLAB of each color patch is computed and compared against the IdealLAB. This results in a dE value for each patch.

The star rating for Specification: FADGI Documents (Unbound): General Collections is as follows:

Metric: Lightness Nonuniformity

This metric only applies to Colorimetric mode. (There is no corresponding metric in Digital Counts mode)

The MeasuredLAB of each of the 20 corner patch is computed. This results in 20 LAB values. Then the standard deviation of all of the 20 L* values is computed.

The star rating for Specification: FADGI Documents (Unbound): General Collections is as follows:

Calculation of dE and dAB:

Calculating dE

The algorithm used to compute dE can be found here:

http://www.brucelindbloom.com/Eqn_DeltaE_CIE2000.html

and here:

https://en.wikipedia.org/wiki/Color_difference#CIEDE2000

There s a handy on-line tool to check particular values here:

http://www.brucelindbloom.com/ColorDifferenceCalc.html

Calculation of da b:

The calculation of da b is very similar, it just does not use the first term.

From the http://www.brucelindbloom.com/Eqn_DeltaE_CIE2000.html, the calculation of dE is as follows. If you cross out the first term, it becomes the calculation of da b:

Glossary of Raw Data Terms

PixelValue - the average RGB value for the Region of Interest

(For all pixels in the Region of Interest, average all R, G, and B)

XyzD65 - Takes PixelValue from above, and converts it to XYZ as follows:

First use the color profile to convert it to XYZ. But this assumes D50.

So then compute:

for i = 0; i < 3; i++
    xyzD65[i] = xyz[i] * xyzD65White[i] / xyzD50White[i]

where:

    xyzD50White = 96.42, 100.0, 82.49; 
    xyzD65White = 95.04, 100.0, 108.88; 

LAB - Convert xyzD65 to LAB using the equations Bruce Lindbloom gives here:

http://www.brucelindbloom.com/Eqn_XYZ_to_Lab.html

Note that at the bottom of the page he provides different values for constants e and k, stating one value is "Actual CIE standard" and the other is "Intent of the CIE standard". The values by ImageZebra are the "Intent of the CIE standard". The white point used is D65.

Ideal_LAB- This is the ideal LAB value, either Encoded in the 2-D barcode, or known for a given target.

Ideal_RGB- This is the Ideal_LAB value converted to RGB as follows:

- If an embedded profile exists, use it to convert Ideal_LAB to RGB using D50 white point

- If an embedded profile does NOT exist, use AdobeRGB1998 using D65 as the white point!!

Right now it is not clear why the choice was made to use D50 for one, and D65 for the other!! Likely a bug, which needs to be investigated.