Variability instead of means?

#1
Hello, I was thinking about analysing variability instead of means but could not find much for an answer. I am interested in learning more about this so I would appreciate some feedback. I made some R code to demonstrate my question (I can't code well in R, improvements are much welcome too).

Code:
library(dplyr)
library(tidyr)

library(ez)
options(scipen=999)

# Faking data function
make_y_values <- function(N, mean=140, within_sd=1, reps=1){
  sj <- 1:N
  subs <- c()
  ys <- c()
  repeats <- c()
  i=1
  for (s in sj){
    subs <- c(subs, rep(s, reps))
    y <- rnorm(reps, mean, sd=runif(1, 0, within_sd)
    repeats <- c(repeats, 1:reps)
    ys <- c(ys, y)
    i = i+1
  }
  return(data.frame(subject=subs, repeats=repeats, y=ys))
}

pre <- make_y_values(N=10, mean=140, within_sd=10, reps=6)
post <- make_y_values(N=10, mean=140, within_sd=1, reps=6)

pre$session <- rep(0, nrow(pre))
post$session <- rep(1, nrow(post))

df <- rbind(pre, post)

grp <- df %>%
  group_by(subject, session) %>%
  summarise(y_mean=mean(y), y_sd=sd(y))
grp

boxplot(y_mean~session, data=grp)

# ANOVA
library(rstatix)
ANOVA <- anova_test(
  data=df,
  dv="y",
  wid="subject",
  within=c("repeats", "session")
)
print(ANOVA)
Rplot01.png

The example experiment might be something like an intervention for blood pressure treatment. We have 20 patients come into our clinic at time 1 (before drug). Their blood pressure got measured 6 times, showing a spread of values with a mean and SD. Patients go home and take our magic drug for some time before coming back at time 2 for another 6 measures. So we now have pre drug, post drug recordings of means and SD in blood pressure. And surprise, the blood pressure means did not differ so we say the treatment failed. But then we spot the variances or SDs look very much different - much smaller at time 2. This seems useful, but I don't know, how would we analyse something like this? ANOVA does not capture it.

I think we can use an F test to compare, but is it useful? Also what if we have a more complicated design (that I am not able to code). What is the approach then?
 

Karabiner

TS Contributor
#2
In your experiment, you have n=20 patients with 2*6 measurements of the outcome.
You could aggregate these measurements patient-wise, according to what might clinically
be important, and use the aggregate value your dpendent variable (for example, mean
blood pressure, SD of blood pressures, difference between highest and lowest blood pressure,
maximum blood pressure...). That variable can be compared pre-post.

With kind regards

Karabiner
 
#3
In your experiment, you have n=20 patients with 2*6 measurements of the outcome.
You could aggregate these measurements patient-wise, according to what might clinically
be important, and use the aggregate value your dpendent variable (for example, mean
blood pressure, SD of blood pressures, difference between highest and lowest blood pressure,
maximum blood pressure...). That variable can be compared pre-post.

With kind regards

Karabiner
Thank you for the idea. If I aggregate and end up with the SD can this be analyzed as if we are just analyzing the mean? e.g. t-test, ANOVA, so on. Or is this a violation of normality? I mean is the population standard DEV assumed to be a normal distribution? It feels a bit strange to think about. Thanks
 

Karabiner

TS Contributor
#4
Thank you for the idea. If I aggregate and end up with the SD can this be analyzed as if we are just analyzing the mean?
e.g. t-test, ANOVA, so on. Or is this a violation of normality? I mean is the population standard DEV assumed to be a normal distribution? It feels a bit strange to think about. Thanks
Sorry, but I do not understand what you mean. How does normality come into play here?
I suggested to chose an outcome variable based on the 6 intra-individual measures pre and post,
respectively, so that a subject has data on "average blood pressure out of 6 measurements" pre and post
(as you already did before, if I understand it correctely), or on "maximum blood pressure out of
6 measurements" pre and post, or or on "SD of intra-individual blood pressure" etc.
Then you can for example ask the question "does intra-individual variability of blood pressure
change between pre and post", and chose an appropriate test (Wilcoxon signed rank
test, or sign test, or dependent samples t-test, or whatever).

With kind regards

Karabiner
 
#5
Sorry, but I do not understand what you mean. How does normality come into play here?
I think it is relevant because if I were to do a t-test (or ANOVA in a more complex scenario) to compare pre/post then this requires normality. Means might be fine, but when I was thinking on this, the variance cannot be normal, and I am not sure if the SD would be either.
 

Karabiner

TS Contributor
#6
I think it is relevant because if I were to do a t-test (or ANOVA in a more complex scenario) to compare pre/post then this requires normality.
A dependent sample t-test requires normality of the differences between pre and post, in the population from
which the sample was drawn. If the sample size is large enough (about n > 30 or so), the test is considered
robust against non-normality in the population. If the sample size is small, then normality should be fulfilled,
but usually cannot be demonstrated, so maybe a "non-parametric" test would be preferable.

But all this is independent of whether you use the individual SDs as dependent variable. You cannot say
beforehand whether the differences between pre and post follows a normal distribution (in the population)
or can you?

With kind regards

Karabiner
 

hlsmith

Less is more. Stay pure. Stay poor.
#7
Two comments, looking solely at variability negates if the distribution moved. Also, if you have multiple observations per subject, you need to control for those otherwise you are not controlling for dependence between them. Multilevel modelling allows you to report the between and within variability explained.

Also, there is a lot of ruckus around how best to examine repeated time points. The standard is to control for the baseline value, since someone with a 190 SBP has alot more room for change than a person with an 80 SBP. Though this depends on what your target study question may be.
 
#8
Thank you for feedback.
But all this is independent of whether you use the individual SDs as dependent variable. You cannot say
beforehand whether the differences between pre and post follows a normal distribution (in the population)
or can you?
Well, I do see your point. I thought SD might (like variance) necessarily be not normal. I think we can say that any distribution of variances would not be normally distributed (as it is always positive, and we square to get there), so thought this might also be true for SD but perhaps not, I did quite some simulation and noticed that they do appear to be normal (though variances definitely are not and we can say this without thinking).

Two comments, looking solely at variability negates if the distribution moved. Also, if you have multiple observations per subject, you need to control for those otherwise you are not controlling for dependence between them. Multilevel modelling allows you to report the between and within variability explained.

Also, there is a lot of ruckus around how best to examine repeated time points. The standard is to control for the baseline value, since someone with a 190 SBP has alot more room for change than a person with an 80 SBP. Though this depends on what your target study question may be.
Thank you for your feedback hlsmith, you are correct it might be nice also to see if the distribution moved. But I do not understand what you mean by controlling for subjects - isn't this done by repeated measures t-test or ANOVA? Could you maybe explain a little bit more about what you mean by controlling for multiple observations per subject? If you mean about the 6 observations in each time point (in my example) then as Karabiner suggests I would be aggregating.

Maybe a normalization subject-by-subject would help to overcome the problem with baseline values, or not? I am not very familiar with the concept of controlling for baseline value beyond throwing all my data onto a repeated measures test.