Rescharts Slanted-Edge Modules Part 3: Edge Results

Imatest Rescharts slanted-edge modules perform highly automated measurements of several key image quality factors using specially-designed test charts. The user does not need to manually select Regions of Interest (ROIs).

This page covers results that are derived from the slanted-edges (i.e., not from grayscale, color, or wedge patterns). It also covers text output (CSV and JSON) files.

• Sharpness, expressed as Spatial Frequency Response (SFR), also known as the Modulation Transfer Function (MTF), can be displayed in several ways for individual edges or from the entire pattern,
• Lateral Chromatic Aberration
• Information capacity

Other results, not derived from slanted-edges are covered in Part 4.

• • Noise (best in eSFR ISO), 
  • Distortion  (differing detail in different modules; best with SFRplus and eSFR ISO. Described in detail here.
  • Tonal response*  (again, with less detail than Stepchart; no noise statistics)
  • Color accuracy*  when used with an SFRplus that contains the optional color pattern, located above the central square.
  • ISO sensitivity*  (Saturation-based and Standard Output Sensitivity) for charts with a grayscale pattern, when incident lux is entered.
  • Vanishing resolution, aliasing, and Moiré from Wedge patterns in eSFR ISO
  • Image, Geometry, Distortion, Field of View  for all charts

*only for charts with tone and/or color patterns. Not in SFRreg (without center chart) or Checkerboard.

Results table – Tangential and Sagittal edges – Multi-ROI summary – MTF asymmetry – Edge and MTF – Chromatic Aberration –  Acutance/SQF – Histograms & noise – Image, Geometry, Distortion, FoV – 3D Plots – Lens style MTF plot – Edge roughness – Point Spread Function – Information capacity – Summary – CSV & JSON output – Links

 

This page combines results for the four Rescharts slanted-edge modules with automatic region detection,  SFRplus, eSFR ISO, SFRreg, and Checkerboard.  These modules contain many common results, making. In many cases we will use SFRplus as an example. Other module produce similar results.

 

New in Imatest 23.1 Information capacity can be calculated from slanted edges. Equations and Algorithms – Instructions – Detailed PDF document   New in Imatest 2020.2  Greatly improved 3D and multi-image displays. The Image Contour map is of considerable interest. Maps of Shannon information capacity and Mean ROI pixel level (related to uniformity of illumination) can be displayed. Imatest 5.1  Chart MTF compensation has been added to compensate for chart limitations. This can double the maximum megapixel capacity of most charts.

 

Four introductory pages —  SFRplus, eSFR ISO, SFRreg, and Checkerboard — describe each module and explain how to obtain and photograph the chart.

Rescharts Instructions Part 2 shows how to run the modules inside Rescharts and save settings for automated runs.

Results

When calculations are complete, results are displayed in the Rescharts window, which allows a number of displays to be selected. The following table shows the figure that displays specific results. Displays are numbered, but the numbers are different for different modules. Results from measurements other than slanted-edge are in Part 4.

Slanted-edge Measurement  (this page)	Display	Module
MTF (sharpness) for individual regions	Edge and MTF	All
MTF (sharpness) for entire image	Multi-ROI summary 3D plot Lens-style MTF plot	All
Lateral Chromatic Aberration	Chromatic Aberration	All
Acutance/SQF (Subjective Quality Factor)	Acutance / SQF	All
Edge roughness	Edge roughness	All
Noise (from slanted-edges)	Histograms and noise stats	All
Point Spread Function (approximate)	PSF plot	All
Information capacity Results (C, NPS, NEQ, SNRi, etc.)	Information capacity plots	All
Other measurement  (Part 4)	Display	Module
Image, Geometry, Distortion, Field of View	Image & geometry	All
Distortion (geometric– radial plot)	Radial Distortion Plot	S+, E, C
Tonal response & gamma	Tonal response & gamma	E+
Detailed eSFR ISO noise	See Color/Tone and eSFR ISO Noise	E+
Color accuracy	a*b* Color error Split colors	E+
ISO Sensitivity	Tonal response & gamma	E+
Uniformity/Light falloff profiles	Color & lightness uniformity profiles	S+, C
EXIF data	Summary & EXIF data	All
Wedge resolution, aliasing, & MTF [eSFR ISO-only]	Wedge aliasing & MTF Multi-wedge plot	E

Legend:  S+ = SFRplus, E = eSFR ISO, E+ = eSFR ISO + SFRreg with center chart, C = Checkerboard

Settings that affect each display will be shown. Most of these settings are not listed in Part 2. They are saved  and used for Auto runs.

Tangential and Sagittal edges

Edge orientation can have a strong effect on MTF:  edges of different orientation (horizontal vs. vertical, etc.) generally do NOT have the same MTF, even if they’re close neighbors. For this reason we often want to limit the edge orientations in multi-region plots to keep them clean and readable. Multi-ROI plots and 3D Plots and 2D Image Contour Map. These plots allow the selections shown in the box on the right.

Plot Vertical-only Plot Horizontal-only Plot both V & H Plot V & H: 2 plots for Auto Plot Tangential MTF-only Plot Sagittal MTF-only

Near-Horizonal and near-Vertical edges (i.e., slanted) edges are familiar to engineers working in imaging. (Pure H and V edges are discouraged because the phase relationship between H and V edges and image pixels can cause significant measurement inconsistency — fixed by (slightly) slanting them, typically by about 5 degrees).

Tangential and sagittal edges familiar to optical engineers (lens designers), but less familiar to most imaging engineers

Sagittal (radial) edges are oriented along the radial line from the center of the image to the edge. Tangential (meridional) edges (shown approximately in the diagram on the right) are oriented normal (perpendicular) to the radial line.

Note that MTF measurements are made in the direction normal (perpendicular) to the edge, resulting in unavoidable confusion. For H and V edges, where we think mostly in terms of edge orientation, this is not much of a problem. But it’s more significant with Sagittal and Tangential edges, which correspond to Tangential and Sagittal MTF, respectively. We have recently improved labeling to try to make this a bit clearer.

Reasonable measurements of sagittal and tangential edges can be obtained even if the edges are not perfectly oriented. The criteria for an edge to be displayed as sagittal or tangential are

Sagittal:      fraction(sagittal edge)        = |sin(α−θ)| > 0.9
Tangential:  fraction(tangential edge)  = |cos(α−θ)| > 0.9

Note that an edge can be either (primarily) horizontal or vertical; it can be sagittal, tangential, or neither (if neither of the above criteria are met).

Some notes: 

• Sagittal and tangential edges are not measured in diagonal patches (relative to the center) because edge angles would need to be near 45 degrees, making good ROI selections difficult.
• SFRplus has more sagittal/tangential squares than other auto-detection slanted-edge charts (eSFR ISO, Checkerboard) because the slants near the corners were designed so that vertical edges would be mostly tangential (to facilitate chromatic aberration measurement).
• We have developed a variant of the SFRplus chart, shown on the right, with 8 total added tangential and sagittal squares in the top and bottom rows. It is entirely software-compatible with the standard SFRplus chart, and is currently available only on request.

Multi-ROI summary display

SFRplus results in Rescharts window: Multiple region (ROI) summary

The multi-ROI (multiple Region of Interest) summary shown in the Rescharts window (above) contains a detailed summary of results for all ROIs. (3D plots also contain an excellent summary.) The upper left contains the image in muted gray tones, with the selected regions surrounded by red rectangles and displayed with full contrast. Up to four results boxes are displayed next to each region. The results are selected the the Display options area on the right of the window, below the Display selection.

The Results selection (right) lets you choose which results to display. N is region number. Ctr-corner distance % is the approximate location of the region. CA is Chromatic Aberration in area, as percentage of the center-to-corner distance (a perceptual measurement). A legend below the image shows which results are displayed. multistyle in the ini file. The View selection (far right) lets you select how many results boxes to display, which can be helpful when many regions overlap. From top to bottom the number of boxes is 4, 3, 2, 2, and 1, respectively. multidisp in the ini file.

Results selection

View selection

The approximate location of maximum MTF, based on a second order fit to MTF results, is indicated by a red ‘O’ at the intersection of vertical and horizontal dotted lines (– – – – O – – – –).

Distortion statistics, shown in the lower left, are described in detail in SFRplus Distortion and Field of View Measurements.

• SMIA TV distortion is the simplest overall measure of distortion. it is positive for pincushion distortion and negative for barrel distortion.
• k₁ (the third order distortion coefficient). r_u = r_d + k₁ r_d³ = where r_u is the undistorted radius and r_d is the distorted radius. r is normalized to the center-to-corner distance. k₁ > 0 for barrel distortion and k₁ < 0 for pincushion.
• h₁ and h₂ (fifth order distortion coefficients). r_u = r_d + h₁ r_d³ + h₂ r_d⁵ A relatively large value of h₂ with a different sign from h₁ may indicate “wave” or “moustache” distortion, where the predominant distortion goes from barrel to pincushion (or vice-versa) as radius increases.

Arctangent/tangent distortion coefficients (used in Picture Window Pro) are also calculated and included in the CSV output file.

MTF asymmetry is calculated in the x (horizontal) and y (vertical) directions. MTF50 as functions of x and y are fitted to second-order (parabolic) curves, which are used to calculate expected MTF50 values at the Top (MTF50T), Bottom (MTF50B), Left (MTF50L) and Right (MTF50R) boundaries of the image (which are further from the center than the actual edges used to calculate MTF).

\(\displaystyle \text{MTF asymmetry}(x) = \frac{MTF50R-MTF50L}{MTF50R+MTF50L}\)

\(\displaystyle \text{MTF asymmetry}(y) = \frac{MTF50T-MTF50B}{MTF50T+MTF50B}\)

Weighted summary results are displayed on the upper-right. The default weights are 1.0 for ROIs in the central region (inside the inner dotted circle), 0.75 for the middle region (between the two dotted circles), and 0.5 for the outer region (outside the outer dotted circle). These weights can be changed in the SFRplus Settings window. The results are independent of the number of ROIs in each region.

Plot settings area

A small number of options are available in the Plot settings area, on the lower right, below the Display box (fewer than for other Rescharts modules).

All displays contain the dropdown menu for selecting the primary channel to analyze. If R, G, B, or Y (Luminance) is selected, all channels are analyzed, but the selected channel is emphasized. There is also an option to analyze any of the channels alone— useful where one of the secondary channels is dark and may cause a run to crash. Luminance (Y) is shown above. If it is changed, results are recalculated.

Many displays allow you to select the ROI for viewing results.

The Edge and MTF display has a dropdown window for selecting the maximum MTF display frequency: 2x Nyquist (the default), Nyquist, 0.5x Nyquist, and 0.2x Nyquist.

 

Edge and MTF display

Edge and MTF display in Rescharts window
Diffraction-limited MTF and edge response are shown as a pale brown dotted lines
when pixel spacing (5.7um for the EOS-40D) has been entered.

This display is identical to the SFR Edge and MTF display. The edge (or line spread function) is plotted on the top and the MTF is plotted on the bottom. The edge may be displayed linearized and normalized (the default; shown), unlinearized (pixel level) and normalized, or linearized and unnormalized (good for checking for saturation, especially in images with poor white balance). Edge display is selected by pressing More settings.

There are a number of readouts, including 10-90% rise distance, MTF50, MTF50P (the spatial frequency where MTF is 50% of the peak value, differing from MTF50 only for oversharpened pulses), the secondary readouts (MTF30 and MTF@30 lp/mm in this case), and the MTF at the Nyquist frequency (0.5 cycles/pixel). The diffraction-limited MTF curve is displayed as a pale brown dotted line.

MTF is explained in Sharpness: What is it and how is it measured? MTF curves and Image appearance contains several examples illustrating the correlation between MTF curves and perceived sharpness.

An important (and optional) readout in the upper plot is

• The Chart contrast (entered in the Display options section of the SFRplus settings & options window),
• The Contrast factor: the ratio between the chart contrast derived from the pixel ratio and the input value of gamma (0.5 in the above display),
• Gamma (from chart): a measurement of gamma derived from the chart pixel levels and the Chart contrast (an input value, as described above). Gamma has a relatively small effect on the MTF measurement, especially for moderate to low contrast targets (10:1 or under).

Chromatic Aberration

Lateral Chromatic Aberration (LCA), also known as “color fringing,” is most visible on tangential boundaries near the edges of the image. Some of the numeric results are grayed out if the selected region (ROI) is too close to the center (less than 30% of the distance to the corner) to accurately measure CA.

The area between the highest and lowest of the edge curves (shown for the R, G, and B channels) is a perceptual measurement of LCA. It has units of pixels because the curves are normalized to an amplitude of 1 and the x-direction (normal to the edge) is in units of pixels. It is displayed as a magenta curve.

Perceptual (area) LCA is also expressed as percentage of the distance from center of the image to the center of the ROI, which tends to better represent system performance: less sensitive to location and pixel count than the pixel measurement. Values under 0.04% of the distance from the center are insignificant; LCA over 0.15% can be quite visible and serious. LCA area can be strongly affected by the demosaicing algorithm.

Information for correction LCA (R-G and B-G crossing distances) is also given in units of % (center-to-ROI) and pixels. LCA can be corrected most effectively before demosaicing. This measurement is relatively insensitive to demosaicing. Results are explained in Chromatic Aberration … plot.

 

Acutance/SQF (Subjective Quality Factor) Acutance and SQF are perceptual measurements of the sharpness of a display (monitor image or print). MTF, by comparison is device sharpness (not perceptual sharpness). Acutance and SQF includes the effects of the human visual system’s Contrast Sensitivity Function (CSF), print (or display) size, and viewing distance (which can be fixed or assumed to be proportional to print height).Acutance and SQF are available when the Acutance analysis checkbox is checked in the More settings window. See Introduction to Acutance/SQF for more detail.

Acutance

Histograms and noise analysis

This display, available when Speedup is unchecked, contains histograms of pixel levels for individual ROIs (original on top and linearized using input gamma on bottom). Since this slows calculations considerably, we only recommend unchecking Speedup when histograms are needed (which will be infrequently).

The black (background) histogram contains pixel levels for the entire ROI. The red histogram is for the right (dark) region, away from the transition, used in the noise statistics calculation. The cyan histogram is for the left (light) region. Sharpening may cause extra bumps to appear in the black histogram. A detailed level and noise analysis is displayed below the image.

Dark, light levels	Original pixel levels normalized to 1 and linearized levels, also normalized to 1
Estimated chart contrast (for gamma = …)	The correct chart contrast must be entered in SFRplus parameters & setup window.
Noise (dark, light, mean; norml pixels %)	RMS noise expressed in pixels, normalized to 100% (for 255 in 8-bit files)
S/N (…; norml pixels)	Signal/Noise, where signal is mean pixel level of ROI at a distance from the transition.
Noise (…; linearized %)	Noise, linearized using gamma (input); normalized to 100%
S/N (…; linearized)	Signal/Noise, linearized

Noise calculations are made in portions of the ROIs (Regions of Interest) away from the transitions. They are facilitated by selecting wide ROIs. The noise spectrum on the lower right contains qualitative information about noise visibility and software noise reduction, which generally reduces high frequency noise below 0.5, typical of demosaicing alone. The region on the right is shown in red; left is shown in cyan. See also Noise.

Image, Geometry, Distortion, FoV

This display allows several aspects of the image to be viewed in detail. It shows the selected regions for MTF/noise analysis (violet rectangles), the yellow rectangles used for generating the uniformity profiles, and the red horizontal curves used to measure distortion). Display options include

Original Image	Extreme Lighten (HSL)
Rec channel	Lighten (HSV)
Green channel	Lighten 1 bit
Blue channel	Lighten 2 bits
Color: Boost Sat. 4X	Original w/5×5 grid
Color: Boost Sat, 10X	Original w/5×7 grid
Lighten (HSL)	Original w/11×15 grid

A number of geometrical results are shown beneath the image. Distortion results are described in more detail in SFRplus Distortion and Field of View measurements

• WxH of image in pixels; m x n squares found; ROI size (referencing input setting), number of ROIs for MTF, etc. (not for profiles).
• SMIA TV Distortion: Barrel (0)
• 3rd order, 5th order, and arctan/tan distortion coefficients The best of these (with the least optimizer error) may be used to find Field of View.
• x,y coordinates of the center of the middle square relative to the image center.
• The average rotation of the pattern in degrees (>0 for clockwise)
• The bar-to-bar vertical height as a fraction of the image height and in pixels.
• Convergence angles (degrees). These are the result of perspective distortion, when the camera is not pointed directly at the target or is misaligned. The horizontal convergence angle (related to yaw) is shown below: calculated by extrapolating the horizontal bars to the top and bottom. A positive angle has a vortex to the left. In the illustration below the pairs of solid red lines at the top and bottom are parallel (to fit the image on the page).

Perfect alignment would be x,y coordinates, rotation, and H,V convergence = 0. Marks may be added to the top and bottom white space of the SFRplus image to facilitate alignment. Vertical convergence angle is derived from lines connecting the right sides of the left-most squares and the left sides of the right-most squares (shown as vertical blue dashed lines – – – above). With fisheye (strongly barrel-distorted) lenses the chart must be precisely centered for the convergence angle measurement to be meaningful.

• measured focal length of the lens is displayed when Bar-to-bar chart height in cm, Lens-to-chart distance in cm, and Pixel spacing (pitch) have been been entered. It is written to the CSV output file if calculated. Equations: Magnification M is bar-to-bar height (on the sensor) / bar-to-bar height (on the chart). For Lens-to-chart distance f₁, lens-to-sensor distance f₂ = f₁M. Using the thin lens equation, focal length f = 1/(1/ f₁+1/ f₂).
• Field of view calculations include the effects of distortion, using the selected distortion model (the best of 3 models is is the default for SFRplus; many models are available in Checkerboard; models are limited in eSFR ISO). See Distortion: methods and modules. FoV in centimeters (cm) is displayed if the bar-to-bar vertical distance (in the chart center for pre-distorted charts) is entered for SFRplus (registration mark vertical spacing for eSFR ISO; square spacing for Checkerboard). FoV in degrees is displayed if the lens-to-chart distance in cm is also entered.

A particularly interesting display is available when you click Crop to ROI (or Crop to 1/2 ROI). The ROI is displayed large enough so its blur is clearly visible and can be compared with Edge and MTF plots and summary results.

Image, Geometry, Distortion, FoV display, showing Crop to ROI, which allows
the edge appearance (enlarged) to be compared to Edge and MTF results.

3D Plots and 2D Image Contour Map

3D plots have been greatly improved* in Imatest 2020.2.  There are now two types of plot. 3D plots display results as three-dimensional pseudocolor images. The new 2D Image Contour Map, shown below, displays results in two dimensions with colored contour lines superimposed on the image.  *Open the Legacy documentation to compare the old results.

3D plot (with image & text) for MTF50P

Region selection should include at least 13 regions. A 5×5 ROI grid (23 regions) was chosen for the analysis on this page. Older charts with two contrast levels (10:1 and 2:1) can be used, but single-contrast charts (usually 4:1) are preferred. Results for Vertical edges are shown. Horizontal edge results can be quite different.

Plotting V+H results together often makes for bumpy plots, which can be difficult to interpret.
For this reason we recommend plotting Vertical and Horizontal edges separately.

The Plot dropdown menu on the right selects the measurement to be displayed. (Only MTF50 with automatic scaling can be displayed in Imatest Studio) Many of the Plot selections are familiar, but a few are new or need explanation. 2, 3 These are Secondary readouts. 12-15 Chromatic aberration results: not plotted near the center of the image, where they are poorly defined. 17. Shannon capacity Approximate for slanted edges. More accurate when measured with Siemens Star patterns. 21. Gamma (from edge) A measure of contrast, derived from slanted-edges. 22. Mean ROI level Measures illumination nonuniformity. Affected by the lens and light source. Only provides useful lens information when an evenly-illuminated lightbox is used (with a transmissive chart). Several Display options are available, shown on the right (set in the display area on the right of the Rescharts window). 3D displays can be rotated for enhanced visualization when Rotate 3D is displayed in the small dropdown menu below the thumbnail image on the upper-right.The (3D) Pseudocolor shaded plot with image and text shown above has been greatly improved in 2020.2.  3D image results can be inverted to reveal detail obscured in a normal display. The background color can be varied from white to black using the Backgnd slider. The default is light gray (0.9). Manual or Automatic scaling may be selected in Master using the Scaling button. Automatic is the default. When Scaling is pressed, the Manual scaling window, shown on the right, lets you set the scaling for the parameter selected in the Plot menu (MTF50P in this case). Minimum and Maximum only apply for manual scaling, i.e., when Auto scaling is unchecked. 6, 11, 16, 21, or 26 contour lines may be selected. The Top button sets a top (pseudo-2D) view. It toggles between Top and Default. You can rotate the image starting from either setting. Multiple 3D plots can be produced by Auto mode (batch-capable) runs by making appropriate selections in the More 3D plots button in the Auto mode settings window. The Expand 3D & Multi checkbox, shown in the Display area of the figure below (and also in the Settings dropdown menu), expands the data to fill the plot when checked. It makes little difference in this example, but can be very helpful where the regions are concentrated near the center of the image, causing much of the display area to be unused. This setting also affects the Multi-ROI summary display. The 2D Image Contour Map was inspired by USGS topographic (contour) maps that fascinated the author when he was young (example on right): Click on it to view full-sized.

Plot options (Master) Display options Scaling window  USGS topographic (contour) map. Our office is about a mile east of the map boundary.

The new 2D Image Contour Map is particularly nice for visualizing results.  An example is displayed below. Three distinct “optical centers” are shown: center of illumination, sharpness, and distortion. These centers mean little if their corresponding measurement is highly uniform.

Lens-style MTF plot

This plot was designed to produce similar results to MTF plots in the Canon, Nikon, and Zeiss websites.  Up to three plot parameters may be selected with the Secondary readout. Typical choices are MTF at 10, 20, and 40 lp/mm (used by Zeiss) or 10 and 30 lp/mm (used by Canon and Nikon). Pixel spacing (um/pixel, etc.) must entered to obtain lp/mm spatial frequencies.

A minimum of 13 regions is required, though more are better. Recommended region selections are 10. All squares, inner & boundary edges (best 3D map) (for single-tone charts) or 13. All squares, inner & bdry except low contrast (good 3D map) (for two-tone charts). V&H edges (both) should be selected.

Though these plots are similar to the website plots, there are several significant differences.

• Imatest calculates he system MTF, including the sensor and signal-processing. The websites display the optically-measured MTF for the lens-only. Imatest results are comparable, but never identical, to the published lens curves.
• The horizontal (H) MTF curves, derived from vertical edges, are mostly radial (sagittal) near the sides of the image (at large distances from the center). The SFRplus chart was designed for this purpose.
• The Vertical (V) MTF curves, derived from horizontal edges, are mostly tangential (meridional) near the sides.
• The published curves assume perfect centering. Some manufacturers, like Canon, obtain their results from design calculations, not from measured test results (no centering issues— or perhaps they use “cherry-picked” lenses!). Real lenses almost always exhibit some decentering, primarily due to manufacturing variability, which is more visible in other displays, particularly the 3D plots. In the lens-style MTF plot, decentering shows up as a spread in the individual readings (x for Horizontal MTF from vertical edges, • for Vertical MTF from horizontal edges).

The curves are calculated using third-order polynomial regression. With higher-order regression, scatter due to lens decentering often caused the curves to display serious irregularities.

Edge roughness plot

Analyzes edge roughness, which is derived from the difference between the transition center for each scan line and the average edge location on the second order curve that fits the edge centers (for the recommended Imatest calculation, described here) or straight line if ISO standard calculation is checked (not recommended).

Edge roughness is not a recommended measurement. See the note just below this text. 

Edge roughness has two components:

• Random (aperiodic), caused by noise.
• Periodic (also called “jaggies” for jagged edges), caused by aliasing (signal response above the Nyquist frequency, 0.5 C/P). Affected by the demosaicing algorithm. 

Periodic noise has peak spectral response at spatial frequencies corresponding to one or two pixels perpendicular to the edge, where one pixel corresponds to the green channel and two pixels corresponds to the red and blue channels of Bayer color filter arrays. In the Roughness Spectrum (lower) plot these frequencies are 1 and 0.5, respectively. The response peaks at these frequencies are fairly typical.

The periodic noise is calculated by transforming the frequency components at f = 0.5, 1.0, 1.5, 2.0 and 2.5 ( i.e., the aliasing fundamentals and the first few harmonics) back into spatial domain (omitting all other frequency components). The aperiodic noise is calculated by subtracting the periodic noise from the total noise.

Three RMS (σ = standard deviation) roughness values are reported: Total, aliasing (jaggies), and Total-aliasing (aperiodic).

The interpretation of edge roughness is somewhat challenging because there is only limited data for comparison. While the pattern on the right is fairly typical (and expected), we’ve seen many spectra that don’t resemble it at all. Unfortunately edge roughness tends to decrease as sharpness decreases (due to lens aberrations, misfocus, etc.). Although we had hoped that edge roughness would enable us to distinguish excellent and mediocre demosaicing algorithms, the inverse relationship with sharpness reduces its usefulness. For most applications we do not recommend it as an image quality metric.

The edge roughness measurement was developed to measure noise in the presence of a signal, which is significant because bilateral filters, which are widely used in consumer cameras, suppress noise in flat areas but often sharpen the image (boost high frequencies) near contrasty features. This means that noise measured in flat areas (very common in test charts such as the ISO 14524 or ISO 15739) may not be representative of the image sensor. Ultimately edge roughness proved to be disappointing because noise decreased as the image got blurrier. We have solved the problem of measuring noise in the presence of signals using the Siemens Star chart to calculate Shannon information capacity. The new feature will be released in Imatest 2020.1 (February 2020), and will be available in the pilot program before the release.

Point Spread Function (PSF) Plot

Imatest can calculate an imaging system’s approximate Point Spread Function (PSF, which can be interpreted as a two-dimensional Line Spread Function) using a pair of collocated, near-sagittal and tangential slanted edges on SFRplus, eSFR ISO, Checkerboard, and SFRreg targets. This is an approximation, not an exact calculation. An optical lens tester is required for exact PSF measurements for lenses.

To obtain PSF plots, select both vertical and horizontal regions for squares that have edges that are close to sagittal (radial) and tangential in orientation. Such a pair is shown on the right.

PSF plot from two edges in SFRplus.
Click to view full-sized.

Information capacity plots

Information capacity is a fundamental KPI (Key Performance Indicator) that combines sharpness and noise. It is described on Information capacity measurements from Slanted edges: Equations and Algorithms . Instructions are on Information capacity measurements from Slanted edges: Instructions. Here is a typical result, taken from the Instructions page. The following image is one of many available. 

Information capacity results, showing NEQ, Results summary, and Noise image

Other results include Noise Power Spectrum (NPS) and Noise autocorrelation (related to sensor electrical crosstalk) in the top plot, and SNRi, Original crop, and de-binned crop in the lower plot.

Summary

• SFRplus analyzes images of the SFRplus test chart, ideally framed so that there is white space above and below the horizontal bars in the chart, i.e., so neither bar runs off the top or bottom of the image. The white space should be between 0.5% and 25% of the image height. SFRplus can work without the bars, but distortion and field of view are not calculated. The bars may run off the sides of the image. Interfering patterns (bars, etc.) outside the chart should be kept to a minimum.
• eSFR ISO should be framed to the registration marks are well inside the image. Where possible the entire chart should be inside the image. eSFR ISO has less spatial detail than SFRplus, but has wedges and larger tonal patches that allow much more detailed noise analysis.
• SFRreg is highly flexible, and can include individual charts facing the camera with angular fields of view well over 180 degrees.
• Checkerboard uses a simple checkerboard (chessboard) pattern that works over a wide range of distances: chart framing is not critical. 
• Lighting should be even and glare-free. Lighting and alignment recommendations are given in The Imatest test lab.
• The first time any of these modules is run, it should be run through Rescharts. This allows
  • parameters to be adjusted and saved for later use in the automatic version, which is opened with a (module) Auto  button in the Imatest main window.
  • results (listed above) to be examined interactively in the Rescharts window.
• The Auto buttons in the left column of the main Imatest window runs the corresponding module in full automatic mode using settings saved from the most recent Rescharts run. Batches of files can be run.

Table of Rescharts slanted-edge MTF modules. (All also available as batch-capable fixed modules.)
All feature automatic region detection, ISO-compliant MTF (sharpness) analysis, and Lateral Chromatic Aberration calculation.

Feature	SFRplus	eSFR ISO	SFRreg	Checkerboard
Sharpness map detail	High	Medium	Depends on arrangement	High
ISO-standard chart design	–	♦	–	–
Color analysis	♦	♦	*	–
Tonal response (OECF)	♦	♦	*	–
Noise analysis	Limited	♦	* Limited	Limited
Distortion	♦	Limited	–	♦
Geometry	♦	♦	*	♦
Features and recommended uses	Imatest's original automatically-detected chart, in use since 2009. robust and versatile. Some white space recommended above and below top and bottom bars. More spatial and distortion detail than eSFR ISO.	ISO-standard chart design. Includes wedge analysis. Supports detailed noise analysis.	Several individual charts are typically placed around the image field. Works with -extreme fisheye lenses (>180º) -the Optikos Meridian system. -Charts at different distances to test focus & depth of field. -extreme high resolution cameras (>36MP) *Available with SFRreg center chart in Imatest 5.0+. FoV is omitted from geometry.	Relatively insensitive to framing: can zoom in or out as long as distance is large enough so chart quality is not an issue and there are detectable corners.

♦ denotes strong support;  – denotes no support.

---

Output files

Result file names— The roots of the file names are the same as the image file name. The channel (Y, R, G, or B) is included in the file name. If a Region of Interest has been selected from a complete digital camera image, information about the location of the ROI is included in the file name following the channel. For example, if the center of the ROI is above-right of the image, 20% of the distance from the center to the corner, the characters AR20 are included in the file name.

(default location: subfolder Results)

Output files for filename.jpg (Y-channel)

Excel .CSV (ASCII text files that can be opened in Excel)
SFR_cypx.csv	(Database file for appending results: name does not change). Displays 10-90% rise in pixels and MTF in cycles/pixel (C/P).
SFR_lwph.csv	(Database file for appending results: name does not change). Displays 10-90% rise in number/Picture Height (/PH) and MTF in Line Widths per Picture Height (LW/PH).
filename_YA17_MTF.csv orfilename_nn_MTF.csv	Excel .CSV file of MTF results for this region (designated by location (YA17) or sequence (nn = 01,...). All channels (R, G, B, and Y (luminance) ) are displayed.
filename_Y_multi.csv	Excel .CSV file of summary results for a multiple ROI run.
filename_Y_sfrbatch.csv	Excel .CSF file combining the results of batch runs (one or more files) with multiple ROIs. Only for automatic SFRplus (not Rescharts). Used as input to Batchview.

Excel .CSV (Comma-Separated Variables) and JSON output

Imatest SFR creates or updates output files for use with Microsoft Excel. The files are in CSV (Comma-Separated Variable) format, and are written to the Results subfolder by default. .CSV files are ASCII text files that look pretty ugly when viewed in a text editor:

File ,Date/time ,PH,Ch,H/V,10-90U,10-90C,Over-,Over-,MTF50U,MTF50C,MTF,Camera,Lens,FL,f-stop,Loc,Misc.,,,,,/PH,/PH,shoot%,sharp%,LW/PH,LW/PH,Nyq,,,(mm),,,settingscanon_eos10d_sfr.jpg,2004-03-19 22:21:34, 2048,Y,H, 1422, 1447, 19.5, -0.7, 1334, 1340,0.154,,,,,canon_g5_sfr.jpg,2004-03-19 22:24:30, 1955,Y,H, 1973, 1301, 48.0, 21.3, 1488, 1359,0.268,Canon G5,—,14,5.6,ctr,sigma_sd9_sfr.jpg,2004-03-19 22:27:55, 1504,Y,H, 1432, 1676, 2.4, -7.7, 1479, 1479,0.494,,,,,sigma_sd10_sfr.jpg,2004-03-19 22:28:32, 1504,Y,H, 1563, 1628, 11.9, -2.0, 1586, 1587,0.554,,,,,

But they look fine when opened in Excel.

.CSV files can be edited with standard text editors, but it makes more sense to edit them in Excel, where columns as well as rows can be selected, moved, and/or deleted. Some fields are truncated in the above display, and Date/time is displayed as a sequence of pound signs (#####...).

The format can be changed by dragging the boundaries between cells on the header row (A, B, C, ...) and by selecting the first two rows and setting the text to Bold. This makes the output look better. The modified file can be saved with formatting as an Excel Worksheet (XLS) file. This, of course, is just the beginning.

It's easy to customize the Excel spreadsheet to your liking. For example, suppose you want to make a concise chart. You can delete Date/time (Row B; useful when you're testing but not so interesting later) and Channel (all Y = luminance). You can add a blank line under the title, then you can select the data (rows A4 through J7 in the image below) and sort on any value you choose. Corrected MTF50 (column I) has been sorted in descending order. Modified worksheets should be saved in XLS format, which maintains formatting.

There are no limits. With moderate skill you can plot columns of results. I've said enough. ( I'm not an Excel expert! )

Summary .CSV and JSON files for MTF and other data

An optional .CSV (comma-separated variable) output file contains results for MTF and other data. Its name is [root name]_[channel location]_MTF.csv, where channel is (R, G, B, or Y) and the location BL75 means below-left, 75% of the distance to the corner (from the center). An example is Canon_17-40_24_f4_C1_1408_YBL75_MTF.csv. Excerpts are shown below, opened in Excel.

A portion of the summary CSV file, opened in Excel

The format is as follows:

Line 1	Imatest, release (1.n.x), version (Light, Pro, Eval), module (SFR, SFR multi-ROI, Colorcheck, Stepchart, etc.).
File	File name (title).
Run date	mm/dd/yyyy hh:mm of run.
(blank line)
Tables	Separated by blank lines if more than one. Two tables are produced.
	The first table contains MTF. The columns are Spatial frequency in Cy/mm, LW/PH, MTF (selected channel), MTF (Red), MTF (Green), MTF (Blue), MTF (Luminance = Y). (...) represent rows omitted for brevity.
	The second table contains the edge. Columns are x (location in pixels), Red edge, Green edge, Blue edge, Luminance (Y) edge, and Chromatic Aberration (the difference between the maximum and minimum).
(blank line)
Additional data	The first entry is the name of the data; the second (and additional) entries contain the value. Names are generally self-explanatory (similar to the figures).
(blank line)
EXIF data	Displayed if available. EXIF data is image file metadata that contains important camera, lens, and exposure settings. By default, Imatest uses a small program, jhead.exe, which works only with JPEG files, to read EXIF data. To read detailed EXIF data from all image file formats, we recommend downloading, installing, and selecting Phil Harvey's ExifTool, as described here.

This format is similar for all modules. Data is largely self-explanatory. Enhancements to .CSV files will be listed in the Change Log.

The optional JSON output file contains results similar to the .CSV file. Its contents are largely self-explanatory. It is stored in [root name].json. JSON files can be read into nearly any programming language with a single line of code using programs available on json.org.

An optional .CSV file is also produced for multiple ROI runs. Its name is [root name]_multi.csv.

Links

How to Read MTF Curves by H. H. Nasse of Carl Zeiss. Excellent, thorough introduction. 33 pages long; requires patience. Has a lot of detail on the MTF curves similar to the Lens-style MTF curve in SFRplus. Here is an interesting list of Zeiss technical articles. 

Understanding MTF from Luminous Landscape.com has a much shorter introduction.

Understanding image sharpness and MTF A multi-part series by the author of Imatest, mostly written prior to Imatest's founding. Moderately technical.

Bob Atkins has an excellent introduction to MTF and SQF. SQF (subjective quality factor) is a measure of perceived print sharpness that incorporates the contrast sensitivity function (CSF) of the human eye. It will be added to Imatest Master in late October 2006.

Optikos makes instruments for measuring lens MTF. Their site has some interesting articles, for example, How to Measure MTF.