Gage R&R After Calibration and Buy-Off: A Worked Example

Calibration first, then GRR

Calibration and Gage R&R answer different questions, so passing one does not prove the other.

The usual order is calibration first, then GRR. If you run GRR on a tool with an uncorrected offset, you can get an excellent precision result for a tool that misjudges every part by the same amount.

Two related checks sit outside this article. Linearity asks whether bias changes across the measuring range. Stability asks whether the tool drifts over weeks or months. If your site runs several tools on the same characteristic, tool-to-tool matching is a separate study as well.

Core formulas

Gage R&R splits the observed variation into a part (or site) component and a measurement component. All standard deviations below are in the unit of the measured characteristic, µm in this example.

Three indices are read from these components:

%GRR compares measurement variation with the variation in your study samples. %Tolerance compares it with the specification width, so it answers the accept/reject question directly. ndc (number of distinct categories) estimates how many groups of parts the system can tell apart within the sampled range.

The guidance most engineers quote comes from the AIAG Measurement Systems Analysis (MSA) Reference Manual, 4th edition (2010):

Treat these as guidance, not universal limits. Your customer or internal procedure may set stricter criteria or decide which index governs. The 4th edition uses 6σ for %Tolerance; some older procedures and software used 5.15σ, so check which multiplier your procedure specifies before comparing results.

Worked example: copper-pillar height after calibration

The tool has just been calibrated. Before releasing it to production, the team runs a GRR study on copper-pillar height, using the same fictional requirement as our PFMEA guide: 48–52 µm, so the tolerance width is 4 µm.

Study design

This gives 10 × 3 × 3 = 90 readings. As noted above, it is a minimal teaching layout, not a recommended plan.

Two design choices matter more than the formulas. Reloading and refocusing between trials captures what really happens in production; repeating a reading without moving the sample makes repeatability look better than it is. Randomising the site order stops operators from anchoring on their previous reading.

The data

Fictional readings in µm. A1 = operator A, trial 1, and so on.

Site,A1,A2,A3,B1,B2,B3,C1,C2,C3
1,49.27,49.37,49.32,49.48,49.67,49.48,49.37,49.46,49.31
2,49.66,49.69,49.68,49.70,49.83,49.96,49.48,49.72,49.64
3,49.89,50.15,50.03,50.01,50.14,50.19,49.87,49.96,49.88
4,50.16,50.24,50.03,50.29,50.22,50.28,49.98,50.04,50.07
5,50.37,50.40,50.15,50.29,50.44,50.17,50.19,50.23,50.36
6,49.68,49.58,49.57,49.63,49.81,49.61,49.49,49.56,49.51
7,50.37,50.28,50.39,50.50,50.67,50.61,50.35,50.42,50.32
8,49.96,49.85,49.91,49.81,49.97,49.77,49.63,49.80,49.74
9,50.65,50.86,50.55,50.69,50.77,50.80,50.47,50.47,50.56
10,50.08,49.93,50.02,50.18,50.07,50.20,49.93,50.11,50.03

Operator means: A = 50.003 µm, B = 50.108 µm, C = 49.932 µm. Site means range from 49.41 to 50.65 µm.

ANOVA table

Variance components come from the mean squares (p sites, o operators, n trials; negative estimates are set to zero):

Some software drops the interaction term when it is not significant and pools it into repeatability. Here the interaction is small, so the conclusion does not change.

Results

ndc = 1.41 × 0.383 / 0.128 = 4.2, truncated to 4.

The percentage columns do not add up to 100%. Standard deviations do not add; only variances do.

Reading the result

1. Against the tolerance, the tool is marginal but usable. %Tolerance is 19.1%, inside the 10–30% band. If your procedure judges release by %Tolerance, the tool can be accepted for pass/fail decisions with an improvement action recorded.

2. %GRR and ndc look like failures, but the sample is the reason. %GRR is 31.6% and ndc is 4. That happens because the 10 sites span only about 1.2 µm, while the tolerance is 4 µm. The study's "part variation" is small, so the measurement system looks large next to it. This says more about the sample than about the tool.

Which index governs should be decided in the procedure before the study, not after seeing the numbers. For product acceptance, %Tolerance answers the question directly. If the tool will also feed SPC or capability studies, its variation must be judged against process variation, and %GRR matters too.

3. Repeatability and reproducibility contribute about equally. Repeatability (σ = 0.090 µm) points to the tool, fixture, focus or sample handling. Reproducibility (σ = 0.091 µm) comes almost entirely from one source: operator B's session reads about 0.1 µm higher than A and 0.18 µm higher than C, on nearly every site. The interaction is small (p = 0.26), so this is a consistent shift, not a problem with particular pillars.

Because each operator measured in a different session, the data cannot say whether the shift came from operator B's technique or from the tool's condition during that session. The next section covers how to separate them.

What this does to capability

Measurement variation adds to every capability estimate made with this tool. Suppose production data shows σ_observed = 0.40 µm. The process variation is closer to:

Cp moves from 4 / (6 × 0.40) = 1.67 to 4 / (6 × 0.379) = 1.76. The gap grows quickly as σ_GRR approaches σ_observed. This correction only holds if the GRR conditions match production use. Before reporting any index, check whether Cpk or Ppk fits your data; the SPC & Capability Analyzer can estimate both from the same dataset.

When the study uses one sample and mixed sessions

Many fabs qualify a tool with one reference sample, measured by different operators in different sessions. That design is practical and close to real use. It has three consequences you should state in the report.

Choose sites that span the range

With one sample, the sites are your "parts". If they all sit near the nominal value, %GRR and ndc will look poor even on a good tool, as the example showed. Where possible, pick sites that cover the expected spread, such as centre and edge, or dense and isolated pattern areas. If the sample cannot cover the range, state in the procedure that %Tolerance governs the decision.

If you can only measure one site

With a single site there is no part variation, so %GRR and ndc cannot be calculated. The study becomes closer to a Type 1 gauge study, which repeats one reference part many times and judges bias and repeatability against the tolerance (indices Cg and Cgk). This method comes from the Bosch and VDA 5 tradition, not from the AIAG MSA manual. The repeat count, multiplier and limits differ between editions and companies, so confirm them in the edition your procedure cites.

Operator and session are confounded

When each operator measures in a different session, the operator effect and the time effect are the same column of data. For release decisions this is acceptable, and arguably realistic: it measures what changes when a different person uses the tool at a different time.

The limitation appears only when the result is poor. A large reproducibility term then says "different person plus different time" matters, but not which one. In the example, operator B's shift of about 0.1 µm could be focus technique, or tool warm-up and room temperature in that session.

To separate them, run a short follow-up:

Keep the conclusion of the original study at the level the data supports. "Reproducibility between sessions with different operators is 0.091 µm" is defensible. "Operator B needs retraining" is not, until the follow-up says so.

Common mistakes

GRR study template

Copy and adapt these three parts. Field names are a starting point; align them with your own procedure and customer requirements.

Part 1 — Study plan (fill in before measuring)

Part 2 — Data sheet

One row per site, one column per operator and trial. Record values at full tool resolution.

Site,A1,A2,A3,B1,B2,B3,C1,C2,C3
1,,,,,,,,,
2,,,,,,,,,
3,,,,,,,,,
4,,,,,,,,,
5,,,,,,,,,
6,,,,,,,,,
7,,,,,,,,,
8,,,,,,,,,
9,,,,,,,,,
10,,,,,,,,,

Add columns for additional operators, sessions or trials as your procedure requires.

Part 3 — Results and decision

Practical recommendations

A GRR result is most useful when it points to an action. Use the component breakdown, not just the headline percentage:

Three habits make the study stand up to review. Fix the governing index and limits before measuring. Write the measurement point and handling rules into the procedure. State any confounding in the report.

A GRR study is a snapshot. Between studies, measuring a check standard on a schedule and plotting it on an I-MR chart shows whether the tool stays stable. That record is also the evidence a PFMEA needs when it claims a height measurement is an effective detection control.

Sources and further reading

• AIAG — Measurement Systems Analysis (MSA) Reference Manual, 4th edition (2010). Source of the %GRR, ndc and acceptance guidance used here. This guide does not reproduce the manual's tables or replace its full methodology. Purchase: aiag.org.
• NIST/SEMATECH e-Handbook of Statistical Methods — Measurement Process Characterization. Free background on repeatability, reproducibility, gauge studies and check standards. (itl.nist.gov)
• VDA Volume 5 and Bosch Booklet 10 — Type 1 gauge study. Referenced for the single-part case only.

Related on Kitstead Labs: PFMEA: A Practical Guide for Process Engineers · Cpk vs Ppk · I-MR Chart Guide · SPC & Capability Analyzer