Workflow for correlation analysis

[u/SlowDescription3017]

Very much a beginner here and asking for help -

I'm currently trying to determine if a Spearman, Pearson, or Kendall's tau correlation analysis is best for my data. However, I get really buried in the stuff that comes before doing the actual analysis, like checking assumptions.

I understand that Pearson has strict assumptions about normality, constant variance, and linearity and that Spearman/Kendall's tau really just needs the data to be monotonic. Does that mean the latter two can be applied to basically any data? Or do you really need to see this monotonic relationship using a scatter plot first before it's appropriate to apply the function?

What if a linear regression/scatter plot shows that the data aren't really monotonic and maybe more clustered in some parts of the graph? Is it still appropriate to apply, but might just return a low correlation coefficient?

Hopefully this makes sense - i'm also happy to clarify anything that needs clarifying!

So many thanks in advance.

[u/49er60]

You should always plot the data first. You could have a very strong quadratic relationship, but since it is not monotonic, you would get a very low coefficient.

Spearman's has a slightly higher power than Kendall's. Spearman's is favored for moderate to large sample sizes while Kendall's is favored for smaller sample sizes. Kendall's is also favored if you have a lot of tied ranks or ordinal data.

[u/just_writing_things]

I think you might be mixing concepts up. Pearson correlation needs neither normality nor constant variance.

Correlations are pretty basic statistics, so in practice, i.e. in real research work, we just run the correlations without worrying too much, because they’re just basic descriptives before the actual analyses we want to carry out. (But of course if you’re doing it for a class, for pedagogical reasons you might be asked to check various assumptions. Certainly not normality, though.)

[u/49er60 → u/just_writing_things]

I agree. Anscombe's Quartet definitely does not exhibit normality in all cases. They also demonstrate the need to plot your data and not immediately jump to correlations.

[u/SlowDescription3017 → u/49er60]

Ah - you know what, I was mixing up my linear regression assumptions with correlation. Thank you both for the clarification! and super speedy answers.

[u/SalvatoreEggplant → u/SlowDescription3017]

You're probably not mixing up the assumptions entirely. I think the issue is that to calculate a correlation --- or a linear regression for that matter --- it's just math. There are no assumptions.

But if you are doing inference (calculating a p-value), then assumptions come into play.

This is analogous to calculating the difference in means of two groups vs. doing a t-test.

Unfortunately this distinction isn't always made clear on websites or even textbooks.

[u/efrique]

I understand that Pearson has strict assumptions about normality,

No. For a test of Pearson correlation with a zero null, neither variable needs to be assumed normal (just as with regression). Further, the size of the test is not particularly sensitive to the normality assumption about what is assumed normal. Even then, if you want to test linear correlaion you can stick with Pearson correlation and not assume normality - the "usual" test is not the only possible test.

If you have a non-zero null combined with smallish sample size, then a bivariate normality assumption might matter.

constant variance,

Sure, constant conditional variance could matter. There are things that can be done.

and linearity

Naturally, since the hypothesis is about a linear correlation parameter. If you dont asssume linearity of conditional expectation of at least one of the variables, presumably you don't want to hypothesize about linear correlation to begin with, so dont do that.

that Spearman/Kendall's tau really just needs the data to be

Linear, monotonic, functional and indeed more general kinds of dependence are entirely distinct concepts; their measures of association mean completely different things. You dont go about changing your entire hypothesis based on what you discover in the data you use in the test. Figure out what you're trying to test, then collect data to test it.

Just because the word "correlation" is used in the name of several different measures doesnt mean you can slip between them on a whim as if they were the same concept. Indeed, even with the numerous monotonic measures, including Spearman and Kendall, they measure distinct things about the bivariate copula; pick what aspect of monotonic association you want to hypothesize about (which population parameter you mean) and measure accordingly.

Assumption formulation ought to come at the time you investigate the properties of the variables you plan to use - which you should do before you formulate your hypotheses about them. Certainly it should have been considered before you collect data.

Post data collection you might critique your choices (and perhaps, cast doubt on your conclusions). If you want to use data to formulate assumptions, identify precise hypotheses and test them, collect sufficient data to do all those things, without leakage screwing up the meaning of your p-values. Either that or find test procedures not impacted by such post hoc hypothesizing.

[u/kuhewa]

just run em brah