Correlation QC thresholds¶
CloudSprite’s default cut is:
threshold = mean(scores) - 2 * std(scores)
outlier = score < threshold
scores are the off-diagonal mean Pearson r values. σ = 2 is a starting point, not a physics constant.
When to change σ¶
| Situation | σ |
|---|---|
| First look at a new product family | 2 |
| Tight process, you already know the batch is homogeneous | 2.5–3 (fewer flags; catch only the tail) |
| Hunting a known bad fixture among similar parts | 1.5 (more flags; review them all) |
Publish the σ you used as a parameter on the average dataset (qc_sigma=2).
Fixed r floors vs batch-relative cuts¶
A floor such as “r < 0.90 is always an outlier” fights the data when the whole lot is noisy, and misses a 0.97 oddball in a 0.995 family. Prefer the batch-relative 2σ cut, and also print the raw scores so a human can apply a floor if the spec demands one.
Degenerate batches¶
- n < 3: refuse. You cannot estimate a std from two pairwise numbers in a useful way.
- std ≈ 0: every trace is identical; there are no outliers. Do not divide by a numerically zero std and flag at random.
- All flagged: σ too tight, or mixed populations (two lots in one filter). Split the filter; do not lower σ until the set is one family.
What correlation does not catch¶
Pearson r is invariant to affine y-shifts and scales. A trace that is the right shape but 3 dB low can still score high. If the spec is an insertion-loss window, apply that window in addition to correlation — correlation is a shape screen, not a limit line.