Crud-Aware Test

Compare your observed correlation against the domain's background dependence, not against zero

Sample correlation
Number of observations
Crud scale (Fisher z)
Typical values:
Classical test
\(H_0\!: \rho = 0\)
Crud-aware test
\(H_0\!: z_{\text{true}} \sim \mathcal{N}(0, \sigma_{\text{crud}}^2)\)

How it works

Both tests use the Fisher transformation \(z = \operatorname{atanh}(r)\), which has sampling variance \(\approx 1/(n-3)\).

Classical test asks: is \(r\) different from zero?

$$z_{\text{classical}} = \frac{|\!\operatorname{atanh}(r)|}{\sqrt{1/(n-3)}}$$

Crud-aware test asks: is \(r\) unusual relative to the domain's background?

$$z_{\text{crud}} = \frac{|\!\operatorname{atanh}(r)|}{\sqrt{\sigma_{\text{crud}}^2 + 1/(n-3)}}$$

The denominator of the crud-aware test includes both sampling noise and background variance. A correlation must exceed the crud scale to be declared significant, not just exceed zero.

The crossover sample size where sampling variance equals crud variance is:

$$n^* = \frac{1}{\sigma_{\text{crud}}^2} + 3$$

Below \(n^*\), both tests behave similarly. Above \(n^*\), the classical test starts flagging crud as "significant."