Very High P Values in Neurofeedback

[tristansguigna]

Hello Everyone,

I am in a bit of a pickle, as I am involved in research on neurofeedback and am conducting a study on whether a ratio of electrical power (microvolts) decreases as a subject continues sessions of the neurofeedback. I have performed regression analysis on each subject (30 subjects) in my study and have gotten very low X variable coefficients (-.0145, -.0022). The X values represent session number (each participant completed 40 sessions) and the Y values are the different ratios of power (i.e. 3.2, 4.5). My P values are also very high for each X variable coefficient as well (0.16, 0.26). To put it frankly, is this data doomed and not able to be discussed in a research paper considering there were hardly any significant decreasing trends? I only got low P values for a couple of the participants' regression analysis. Is there something I can do the make the P value lower as I conduct the regression analysis? Is it not valid or sound to have a low P value and also a low X variable coefficient? I am new to statistics so any kind of guidance would help. I was hoping to see this ratio of power decrease in subjects and it's disappointing to see such high P values for every regression analysis. Thank you.

Best,
Tristan Sguigna

 

[Karabiner]

To put it frankly, is this data doomed and not able to be discussed in a research paper considering there were hardly any significant decreasing trends?

Why is it necessary that you present statistically significant findings at all means?
You wrote:
I am conducting a study on whether a ratio of electrical power (microvolts) decreases as a subject continues sessions of the neurofeedback.
So your result is: you have found no evidence that this ratio decreases.
This is not scientifically unsound, unless you have made big mistakes
(e.g. the measurement of the electical power was massively flawed, or
something like that). Of course, there could exist a tiny effect in the
population and your study was just too small to detect.

Your data analysis could be improved, though. Instead of performing
30 separate regression analyses, you could use a multilevel model where
session number and ratio are "nested" in participants. This could increase
statisctical power.

With kind regards

Karabiner

 

[hlsmith]

Yes, it was unclear if you ran a regression for each subject. If so, look into multilevel models like @Karabiner mentioned. Otherwise, your samples are just for 1 person, how generalizable it that? Not very.

Given your wording and doubt in your analytics, I would look into bringing a statistician on board. Multilevel modeling is much more complex then simple regression.

 

[tristansguigna]

Karabiner and hlsmith,

I really appreciate your direct input. I am going to look into multilevel model regression and getting in touch with a statistician. In the meantime, could you both elaborate on the multilevel model a little bit? I already watched some youtube videos but other perspectives help. Thank you.

Best,
Tristan S.

 

[hlsmith]

The basic premise is that you have subjects clustered in groups, that could also mean repeated measurements from the same person. The most basic example is comparing an educational intervention. You have a intervention in a school and you want to examine its impact. However you have students clustered in classrooms so student results within a classroom may be more similar than those between classrooms, so you want to control for this similarity. So now you are controlling for between classroom and within classroom variability.

The other scenario of repeated measures, is that you have a bunch of measurements but people contribute more than one measurement data. You would want to control for the individuals' measurements because they are going to be correlated with each other. If you don't control for measurements being clustered in people you are missing this bit of variability. I primary assumption in most regression models is independence between observations, if you don't have independence then you have to control for it - typically with multilevel models (level one student, level two classrooms, possibly level three school if you have multiple schools).