What analysis do I run?

#1
Hi All,

I am conducting a study within the ASD population that is looking at IQ, executive function and adaptive skills for which of them I am collecting data using different tests. I am looking first at the association between IQ and executive function, then I want to look at the difference in executive function between low and high IQ and also how adaptive skills are associated to executive function, however I am very confused about what analysis I should use. If anyone could help please...

Many thanks!

J.
 

CowboyBear

Super Moderator
#2
Hi there :welcome:

I am conducting a study within the ASD population
ASD = autism spectrum disorder?

I am looking first at the association between IQ and executive function
Correlation and/or simple linear regression?

then I want to look at the difference in executive function between low and high IQ
Seems like the same as the first research question, no?

and also how adaptive skills are associated to executive function
How are adaptive skills measured?
 
#4
Hi!

Thanks a lot for you reply!

Sample size is 25 and yes, Autism Spectrum Disorder!

First I want to see if EF and IQ are correlated and then the difference between groups (High and Low IQ) so I guess they require 2 different analysis.

For Adaptive Skills I have collected y data using the Vineland Adaptive Behaviour Scale so that will be another variable.

Any idea of what should I do?

Again, thanks a lot!!

J.
 

Karabiner

TS Contributor
#5
Any idea of what should I do?
Get a larger sampe sitze, I'd say. You want to perform half a dozen tests or so, and you want to analyse subgoups, all based on just 25 subjects, using noisy data, in a context where a myriad of additional, yet unmeasured, influence factors are present. Common sense tells us that all this has extremely limited virtues. You'll never have the chance to properly which of your positive or negative resuts are false-positive or false-negative. So why bother?!

(I'd recommend to generally forbid such kind of pitifully underpowered studies with patient samples, but probably no professor will listen. )

however I am very confused about what analysis I should use
Your faithful study subjects would maybe have been interested to know this in advance.

With kind regards

Karabiner
 
#6
Thank you for the advice and I would if I was going to conduct a large piece of research, however this is a research project for my Masters and I have been advised to roughly use this sample size. I am not trying to proof anything just yet. I have to use real data and show basically how I conducted my study and analysed the data. Hopefully when I get my degree I can and I will.

J.
 

CowboyBear

Super Moderator
#7
I know this isn't what you want to hear, but I second Karabiner's comments. With a sample size that small you are effectively asking your participants to spend their time on something that doesn't have any research value beyond serving as a learning opportunity for you. That isn't really fair to them (unless you pay them for their time). In a world with the possibility of online data collection, surely it's possible to get a more reasonable sample size? (EDIT: Ok, to be fair, the population you are working with might make this hard - perhaps we're being a bit harsh here. When you say "ASD", is there a particular severity level you're restricting your sample to?)

First I want to see if EF and IQ are correlated and then the difference between groups (High and Low IQ) so I guess they require 2 different analysis.
The correlation is itself looking at the relationship between IQ levels (e.g., high or low) and EF. You would need some kind of strong reason to add an analysis where you dichotomise IQ levels into high and low; ultimately this is just a less powerful way of answering the same question. Dichotomising continuous variables is usually a bad idea.

also how adaptive skills are associated to executive function [...] For Adaptive Skills I have collected y data using the Vineland Adaptive Behaviour Scale so that will be another variable.
Again, just sounds like a correlation.
 
#8
Hi!

To be honest I really appreciate all your comments here but I was really trying to get help because I am a novel in regard to statistics (I am a psychology student)- However I am a teacher too and that is where I am getting my data from so I don't know what this has become almost a criticism to my research study (which is a University Project). I am not being unfair with the participants or trying to get false or poor results, I am trying to learn how to do a Research Study and hopefully in a future I will be able to run a proper study with as many participants as I need. It just came to luck that I am a special needs teacher and I can use that data for my project.

Thanks anyway!

J.
 

Karabiner

TS Contributor
#9
I am trying to learn how to do a Research Study
Research ethics and appropriate sample size are crucial for
proper conduct of a research study.

Within the framework of null hypothesies testing, you'll have to
put a very low limit to the number of analysis you want to perform,
if sampe size is as small as yours.

. I am looking first at the association between IQ and executive function, then I want to look at the difference in executive function between low and high IQ and also how adaptive skills are associated to executive function,
Looks like 2 correlations (Pearson or Spearman correlation) might do the job here,
first between IQ and EF, second between adaptive skils and EF
As CowboyBear already told you, do not dichotomize continuous variables,
at least not for statistical testing. You maybe could perform some descriptive
statistics (without testing) where you display EF in groups with low/middle/high
IQ (low/middle/high IQ not based on the samlke data but on normative
values, I suppose).

With kind regards

K.
 
#11
When I first saw this post I agreed with what was said that this seemed to be fairly uncertain. Then I realized that I did not have a clue how uncertain such a study would be.

Karabiner and CB knows much more than I about social science and psychology, but I believe that someone has said that in psychology a correlation coefficient of 0.40 (or larger) is considered to be a large effect.

So I asked myself: what is the power of test when the sample size is n=25 and the (population) correlation coefficient os 0.40?

(And the power is the probability to get a statistically significant sample correlation coefficient. One want to a high power, 80% or 90%)

So I did this little simulation program in R. R can be freely downloaded and it can be rerun and played with by jeannette88.)

Code:
# install the MASS package first
library(MASS)

n_obs <- 25
rho   <- 0.40
n_rep <- 10000  # number of replicates 

kovar <- matrix(c(1, rho, rho, 1), 2, 2)

# from:  https://stat.ethz.ch/R-manual/R-devel/library/MASS/html/mvrnorm.html
# mvrnorm(n = 1, mu, Sigma, tol = 1e-6, empirical = FALSE, EISPACK = FALSE)

pval <- numeric()
set.seed(210417) 

for(i in 1:n_rep){
  y <- mvrnorm(n=n_obs, mu= c(0, 0), Sigma=kovar,  tol = 1e-6)
  pval[i] <-cor.test(y[,1], y[,2])$p.value
  # print(cor.test(y[,1], y[,2])) # preliminary printing for n_rep=4
  # print(pval[i])
}
hist(pval)

sum(pval < 0.05)
# [1] 5371
# power about 53% (5371 out of 10.000 had a p-value less than 0.05)

it seems like the power is about 53%, so it it not very bad. But it is a little bit of a toss of a coin if the study will be a success or not. Is that OK for the original poster or the professor?