Hvpotheses testing

#1
Hello, please give me instructions with testing the hypotheses:

H1: No relationship between predicted and collected data for model 1.
H2: No relationship between predicted and collected data for model 2.

H1 is tested using predicted model 1 data (pm1) and collected data (cd).
H2 is tested using predicted model 2 data (pm2) and collected data (cd).

Pearson and Spearman correlation is used for testing.

H1 is accepted and H2 is rejected.
Conclusion from this observation is that model 2 is better than model 1.

My question is: can the conclusion can be made on accepted/rejected hypothesis or I need to make new hypothesis that model 2 is better than model 1 and to test pm1 and pm2 depending on cd. If I have to make new hypothesis, which statistic test is preferable to be used when 3 variable have to be compared?
 

Karabiner

TS Contributor
#2
H1 is accepted and H2 is rejected.
Within the frequentist framework, you cannot accept a "no relationship" hypothesis.
"H1 is not rejected" would usually be the appropriate statement.
The non-rejection could be due to H1 (usually a statistical Null hypothesis)
being correct, but more probably, it was not rejected just because the sample
size was too small.

Conclusion from this observation is that model 2 is better than model 1.
No. The differece between significant and non-ignificant is not in itself significant.
https://amstat.tandfonline.com/doi/abs/10.1198/000313006X152649
If you want to compare models, then you have to compare them directely, and test
their difference directely, not indirectely by comparing two separate analyses.
My question is: can the conclusion can be made on accepted/rejected hypothesis or I need to make new hypothesis that model 2 is better than model 1 and to test pm1 and pm2 depending on cd. If I have to make new hypothesis, which statistic test is preferable to be used when 3 variable have to be compared?
This is very abstract and hard to follow. What is the topic of your study
what are the precise reserach questions, how was the study/data collection
designed, which variables were measured and how exactely were they
measured, and how large is your sample size?

With kind regards

Karabiner
 
#3
Tnx Karabiner for your answer.

Topic of study: QoE prediction in telecommunication
Precise research questions: Optimal model for QoE prediction
Data collection: Polynomial regression, Neural network and simulation study: output three vectors pm1, pm2, and cd (all are vectors 140x1 with real numbers between 1 and 5)
Variable measured: pm1 (predicted QoE value as output of model 1), pm2 (predicted QoE value as output of model 2), cd (real measured QoE value - referent)
Exactely measured: pm1 - polynomial regression, pm2 - neural network, cd - output of simulation
Sample size: 140 QoE values

Can I make new vectors: first (cd-pm1) and second (cd-pm2) and test which one is closer to zero? Which test to use in this case?

All the best,
Ida
 
#4
Matched Pairs T-Test is using for testing datasets that have same input points.
ANOVA cant be used because this condition is not satisfied?
 

Karabiner

TS Contributor
#5
Can I make new vectors: first (cd-pm1) and second (cd-pm2) and test which one is closer to zero? Which test to use in this case?
Yes, and dependent sample t-test is what I would choose either.

With kind regards

Karabiner
 
#7
I have one more question. Hypothesis in this case will be formulated as follow:
H0: Prediction errors of model 1 and model 2 have mean difference equal to zero. ???

After application of dependent sample t-test, hypothesis will be rejected or not.
If hypothesis is rejected, which model is better? Which output have this information?