+ Reply to Thread
Results 1 to 2 of 2

Thread: Good model for association of for prediction?

  1. #1
    Points: 504, Level: 10
    Level completed: 8%, Points required for next Level: 46

    Posts
    2
    Thanks
    1
    Thanked 0 Times in 0 Posts

    Good model for association of for prediction?




    Hello everybody, I have a "conceptual issue": I have been told that the program that performs best in finding a lot of feature associated to a phenotype of interest will not perform best for predicting the phenotype of interest out of the features, and vice-versa.
    I do not quite understand that; In my opinion, the more SNPs you are able to associate, the better should be your prediction of what phenotype is which.

    Nevertheless, Google seems to tell me that I have been told the truth; there are methods for prediction and other for association, and this has been confirmed by a statistician I have listened to.
    I have tried to understand that by myself. And these are the elements that I have understood:
    - we do not use the same measure to assess the efficiency of a predictive tool and that of the association tool; for instance, in genome-wise association studies, we compare for each SNP allele frequencies in cases and in controls. From this comparison we calculate a p-value, that assesses the probability that null hypothesis is true given our data, based on a certain distribution, that frequencies should have under the null. Based on this p-value, we decide whether the association is significant or not.
    If we wanted to predict, we would search for the SNP that predicts best the phenotype (case or control). The measure will then relate to whether yo have well classified each individual (no null distributions needed, no pvalues, just an evaluation of how good you did based on your data). It could be the AUC for instance.

    Still why do these 2 measures not give us the same result?
    Wouldn't that SNP that predicts best, be the one that is best associated?

    To this question I am replied: "yes, but the problem is that the signal is blurred through a lot of SNPs, so only 1 SNP may be bad for prediction.A tool that uses more SNPs to predict may perform better". But it's the same for association! A tool that finds more associations will be better!

    I also understand the difference between association and causation (it's the question google keeps answering when I search for this question)

    But could anybody explain me why some methods are better at prediction and other are better in association?

  2. #2
    Fortran must die
    Points: 58,790, Level: 100
    Level completed: 0%, Points required for next Level: 0
    noetsi's Avatar
    Posts
    6,532
    Thanks
    692
    Thanked 915 Times in 874 Posts

    Re: Good model for association of for prediction?


    There are many ways to evaluate a model and commonly they won't generate the same result because they are really asking different questions. For example using negative log liklihood in logistic regression (one way to evaluate the value of a model) is really asking how much better the model is than no predictor at all. On the other hand the Hosmer-Lemeshow test for logistic regression is asking how well the model categorizes individual points (does it predict a data point will be to be in the category it actually is in). Different questions are answered better by different approaches. In addition some models may do a good job of predicting the sample you use and may do a worse job of predicting out of sample data (that is results in a second set of data).

    I don't think anyone without expert knowledge of your topic is going to able to explain your specific question.
    "Very few theories have been abandoned because they were found to be invalid on the basis of empirical evidence...." Spanos, 1995

  3. The Following User Says Thank You to noetsi For This Useful Post:

    scratchscratch (12-29-2014)

+ Reply to Thread

           




Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats