Kitstead Labs
Split-lot analysis
Semiconductor Split-Lot Analysis: Welch t-Test, ANOVA, and Multiple Comparisons
A split lot can generate several groups of measurements, but a comparison is only as credible as its experimental unit and assumptions. This guide works through unequal sample sizes, a two-level Welch comparison and a three-level ANOVA with all-pairs follow-up. It separates a detectable mean difference from an engineering threshold and explains why neither a p-value nor a circle diagram qualifies a process.
Quick answer
For two independent observed levels, a two-sided Welch t-test compares means without assuming equal variances. For three or more levels, ordinary one-way ANOVA tests equality of means under a common-variance model; eligible Tukey–Kramer comparisons identify pairs, including unequal N. Check independence and confounding before interpreting either result.
Start with the experimental unit
A split-lot comparison asks whether the selected response differs across observed conditions. Before choosing a test, record the factor levels, response units, assignment procedure and experimental unit. Repeated measurements of the same wafer are not automatically independent wafer replicates. Treating sites as independent treatment assignments can understate uncertainty.
Randomization, blocking and traceable sampling make an experiment interpretable. If condition A was measured on one tool and condition B on another day and tool, a difference can be confounded with those changes. A small p-value cannot separate causes that the design did not separate.
Two observed levels: a two-sided Welch t-test
Welch's test compares two population means without pooling their variances. Let n₁ and n₂ be usable sample counts, x̄₁ and x̄₂ the means, and s₁² and s₂² sample variances (divisor n−1). For independent observations and a suitable sampling model, test equal means against a difference in either direction.
t = (x̄₁ − x̄₂) / √(a + b)
ν = (a + b)² / [a²/(n₁ − 1) + b²/(n₂ − 1)]
p = 2 × P(Tν ≥ |t|)
Welch handles unequal variance in this two-group comparison; it does not repair dependence, extreme distribution problems in tiny samples or confounding. Paired or blocked observations need a model that reflects that design.
Three or more levels: ordinary one-way ANOVA
Ordinary one-way ANOVA asks whether all k means are equal. It pools residual variance, so independence, approximately normal errors and defensible common variance matter. The Product 02 review-candidate workflow uses this method for 3–12 observed levels. Welch's two-level robustness does not carry over to that ordinary ANOVA branch.
SSbetween = Σnᵢ(x̄ᵢ − x̄grand)²
SSerror = ΣᵢΣⱼ(xᵢⱼ − x̄ᵢ)²
MSE = SSerror / (N − k)
F = [SSbetween / (k − 1)] / MSE
F is dimensionless. Sums of squares and MSE have squared response units. A significant omnibus F indicates a difference somewhere among the means; it does not identify the pairs.
Which pairs differ? Tukey–Kramer comparisons
For eligible comparisons with the same pooled-variance assumptions, Tukey–Kramer supports unequal group counts and accounts for all pairs in the selected family. For groups i and j:
q = |x̄ᵢ − x̄ⱼ| / SEq
Difference interval = (x̄ᵢ − x̄ⱼ) ± qcritical × SEq
Use the studentized-range critical value for k groups and N−k residual degrees of freedom at the chosen family error rate. Here α = 0.05. Significance requires q > q critical. This correction does not cover unrelated factors, responses or repeated exploratory searches.
Worked example: synthetic etch-rate conditions
Assume each number represents a separately assigned, independent experimental unit. Response: etch rate in nm/min. The invented conditions A, B and C have unequal N. For the ANOVA illustration, assume approximately normal errors with common population variance; unequal sample variances below are not themselves proof for or against that assumption. These small samples cannot validate the model.
| Condition | Etch rates (nm/min) | N | Mean | Sample variance |
|---|---|---|---|---|
| A | 98, 100, 102 | 3 | 100 | 4.0000 |
| B | 100, 102, 104, 106 | 4 | 103 | 6.6667 |
| C | 107, 109, 111, 113, 115 | 5 | 111 | 10.0000 |
If the study had only A and B
Using just those seven observations gives a = 4/3 and b = (20/3)/4 = 5/3. Thus t = −3/√3 = −1.73205, ν = 4.95918 and two-sided p ≈ 0.144293. This fails to reject equal means at 0.05; it does not demonstrate equivalence. This is an alternative two-level illustration, not an extra unadjusted post-hoc test for the three-level study.
The complete three-level study
N = 12, grand mean = 1267/12 = 105.583333 nm/min. Within-group sums of squares are 8, 20 and 40, totaling 68. The weighted between-group sum of squares is 266.916667.
| Source | SS | df | MS | F |
|---|---|---|---|---|
| Between conditions | 266.916667 | 2 | 133.458333 | 17.663603 |
| Residual | 68.000000 | 9 | 7.555556 | — |
| Total | 334.916667 | 11 | — | — |
F(2, 9) = 17.663603 gives p ≈ 0.000765728. The null of equal means is rejected under the stated assumptions. With q critical ≈ 3.948492, the all-pairs results are:
| Pair | Mean difference | q | Adjusted p | Interval | q > critical? |
|---|---|---|---|---|---|
| B − A | 3 | 2.020899 | 0.367471 | [−2.861488, 8.861488] | No |
| C − A | 11 | 7.749526 | 0.001017 | [5.395345, 16.604655] | Yes |
| C − B | 8 | 6.135720 | 0.004800 | [2.851796, 13.148204] | Yes |
For example, C−A uses SEq = √[(7.555556/2)(1/5 + 1/3)] ≈ 1.419442 nm/min. Dividing 11 by this quantity gives q ≈ 7.749526. Values are calculated without intermediate rounding; displayed figures are approximations.
Suppose an engineering review set a minimum meaningful increase of 5 nm/min before data collection. Both C comparisons have observed mean increases above 5, but only the C−A simultaneous interval lies wholly above 5. C−B is statistically different from zero without this interval establishing an increase greater than 5. An increased etch rate is also not inherently a better process: profile, selectivity and other responses may constrain it.
Statistical significance and engineering relevance
A p-value measures compatibility with a null model; it is not the probability the process change works. Review effect sizes, uncertainty, specification consequences and plausible causes. Non-significance can reflect imprecision, while significance can concern a difference too small to matter. Neither substitutes for follow-up confirmation.
Marginal comparisons collapse across other conditions. To investigate whether a factor's effect depends on another factor, read DOE interaction effects. An all-factor eta-squared overview describes marginal association, not causal importance or adjustment for the other factors.
Comparison Circles are a visual aid
The numerical table above makes the decision auditable without estimating angles or judging overlap by eye. With unequal N, pair-specific uncertainty changes, making informal visual shortcuts especially risky.
Product status and analysis boundaries
The Semiconductor DOE & Split-Lot Analyzer v0.3.0 is a Windows-first review candidate, not commercially released. Its specified ANALYSIS workflow uses two-sided Welch for exactly two observed levels and ordinary one-way ANOVA for 3–12; eligible Comparison Circles use Tukey–Kramer, including unequal N, with DETAILS authoritative. Release/commercial review is still pending. Current review verification is limited to Windows Excel LTSC 2024; Mac is deferred and untested.
This educational calculation is independent of the candidate workbook. Insufficient usable samples or variation can make inference unavailable; that is not a zero effect. For a different question—monitoring a continuing process against requirements—see capability versus performance.
Sources and further reading
- NIST: Two-sample t-test — Welch statistic and degrees of freedom.
- NIST: One-way ANOVA — sums of squares, degrees of freedom and F.
- NIST: Tukey’s method — simultaneous pairwise comparisons and unequal N.
- SciPy: tukey_hsd — reference implementation used to independently audit the synthetic comparisons.