Logit v. Probit: A fight to the death

Dason

Ambassador to the humans
#41
I am not considering Bayesian here. Because the approach is different. I agree all the points you mention about Bayesian. Also not considering Bootstrap.
Booo. I've been becoming much more bayesian lately...

Check Nelder or Maculloch's GLM book( not remembering which one), they have explained asymptotic distribution of logit. The variance matrix of beta in the form of covariance of weighted least square method. I tried it for probit( long time back) and not able to solve.
I'll check that out when I get time but either way we're looking at asymptotic covariance matrices which you can get from the estimation process. Like I said it's possible to get the standard errors for the probit model.

I would love to see more discussion on the core topic. This would give more take away and less fight.

@Dason, Why the last part "A fight to the death "? any reason?
No reason - I guess I just thought the idea of two link functions fighting to the death was humorous.
 

noetsi

No cake for spunky
#44
Reading the history of regression I noted that probit was "invented" because (when they had no modern computers) it was easier to calculate than logistic regression. Later when they had computers it fell into disfavor because of the ready availability of the odds ratio in logistic regression and the fact that it was easier to explain (to non-statisticians anyhow) what the logit was than probit.
 

terzi

TS Contributor
#45
Why do I never find this threads on time:(:(?

Anyway, I think the "easy to explain" issue is of low importance when selecting a model (or any analysis for that matter). When talking about binary models I try to focus on the expected values, which is what we predict with the model. In this case, that expected value is a probability, so I just skip odds ratios and talk about probabilities, which I think is an easier concept to understand for non statisticians. As far as I know, both logit and probit models are estimated with maximum likelihood methods and you can have probabilities with both, so I personally don't have much of a difference with interpretations (at least when presenting results).

So I'd like to mention the only conceptual reason I know to choose between logistic or probit regression. Probit models can be hard to estimate when the outcome is rare. This means that in cases when the number of "1's" in your dataset is low the model may have problems, even if you have a huge dataset. Logit models, on the other hand, are way better when handling rare outcomes. For reasons I have yet to determine, in my life most datasets have had a really low number of successes (is that the real plural?:confused:). I assume that most studies work in understanding the probabilities of something unusual happening, so logit models may be more appropriate for these particular situations. That is why I think logit is far more common.

By the way, log odds don't have to be necessarily linear. In fact, you have to check for that linearity in your model. If the relationship is not linear, you may have to correct the model, such as you do with OLS, by using adjusted variables or Fractional Polynomials.
 

noetsi

No cake for spunky
#46
Anyway, I think the "easy to explain" issue is of low importance when selecting a model (or any analysis for that matter).
So when you justify your model to the Vice President for Product development (who never heard of chi square) the complexity of your model is not an issue :)

I wish I worked for your managers.
 

terzi

TS Contributor
#47
So when you justify your model to the Vice President for Product development (who never heard of chi square) the complexity of your model is not an issue :)
Well, no because, as I said, I don't explain models, I focus on results. I say something like "the model shows that the probability of purchase goes form 0.10 to 0.90 when changing from campaign A to campaign B". See? No chi-squares anywhere:)

Anyway, I manage an independent consulting firm so I have no bosses, my clients are mostly governments and business that rely in our results, so I probably don't have much of a problem convincing them. Though I agree that those with bosses may have a harder time.
 

noetsi

No cake for spunky
#48
And he doesn't ask you how you got those results? Just takes your word for it?

You are right about being a consultant. My comments were about those who work inside an organization (and don't have the very rare boss who is familiar with statistics).
 

Jake

Cookie Scientist
#49
It seems to me that one of the main reasons for hiring a statistician in the first place is so that they can do the data analyses that you yourself are not capable of doing because you don't understand how they work. If I thought I could understand just fine how the data analyses worked, I wouldn't need to hire a statistician to do them for me. Maybe I'm missing something.
 

noetsi

No cake for spunky
#50
Well first, you are applying logic to something (decision making in organizations) that is not inherently logical (this is my research area). Second, you are commonly hired at one level and report to another. So while your immediate boss might respect (or even understand) what you do the levels above him you report to won't. Third there is the magical degree reality. I got hired because they were impressed I had a doctorate, but that did not lead them to respect the methods used - they were impressed by a PHD despite having little use for the formal data analysis.

I suspect that only a tiny number of statisticians (meaning a doctorate or masters) are hired by private firms outside medicine.
 

Dason

Ambassador to the humans
#51
I suspect that only a tiny number of statisticians (meaning a doctorate or masters) are hired by private firms outside medicine.
What leads you to this idea exactly? I know of quite a lot of places looking for PhD level statisticians outside of medicine.
 

spunky

Can't make spagetti
#52
I suspect that only a tiny number of statisticians (meaning a doctorate or masters) are hired by private firms outside medicine.
i know you dont do it on purpose but sometimes your sweeping generalizations do get to me... for instance, only from people in this board, people who have done stuff outside from academia:

Dason: agriculture
jpkelly: environmental sciences
vinux: business/finance
me: education

... and i'm sure there are others but i just dont remember their posts from the top of my head...
 

noetsi

No cake for spunky
#53
What leads you to this idea exactly? I know of quite a lot of places looking for PhD level statisticians outside of medicine.
Talking to people who worked for many years in business.

Which private firms outside of medicine do you know that are looking for statisticians? Actuerial yes, real statisticians?

I would be delighted if there were many Americans businesses employing statisticians (which I am not). Nothing in the data I have seen suggests this. Government, think tanks, academics is where they largely are.

I imagine that firms contract for them for special activities such as Six Sigma. But that is very different than having them as employees. In honesty I have never seen a single statistician in any firm I worked for - much to my regret.
 

noetsi

No cake for spunky
#54
Dason: agriculture
jpkelly: environmental sciences
vinux: business/finance
me: education
Sweaping generalizations is what I am taught to produce. :)

Of the four only one (finance) is a core private sector business. My point is that line businesses, the vast majority of US firms don't employ statisticians commonly even when they should. When you see them they are in highly specialized areas such as finance, medicine, or engineering - and their influence with senior management is commonly limited. Most senior US managers come out of areas like finance (or marketing) and thus have limited interest in statistics.

My critique of the US business community is (this is the area my doctorate is actually in, organizational performance) that they employ far too few statistically inclined people and show far too little respect for that type of data, particularly among senior managers. Significantly less so then say Japan which is why Deming was largely ignored here and listened to there. And everyone I have talked to (including those working with senior level managers and highly technical fields like finance and engineering) agree. Indeed they laugh at my argument that statistics should be more important (or at any case that it will be).

If they are more common than I think (if say 1/100 of a percent of US firms with at least 1,000 employees employ at least one statistician) that is awesome news. I just don't believe it given the data I have seen.
 

Dason

Ambassador to the humans
#55
I'm assuming you're not counting "Dason: agriculture" under your list of those you consider for private sector business. You should reconsider that. I know of a lot of Ag companies in the local area that hire and use statisticians. Pioneer, for example, has a fairly large staff of full time statisticians designed and analyzing field experiments to test their products. I did consulting for a company that I can't disclose where we were analyzing some data they had just collected - they were planning on hiring some full time statisticians at that point. I've been to numerous conferences where Ag businesses send some of their full time statisticians to to learn more about the methods being used and the advances in software so they can analyze their data better.
 

spunky

Can't make spagetti
#56
same here.... Pearson Pusblishing, CTB-McGraw-Hill, Nelson Publishing... noetsi, all the big names in publishing companies snap statisticians like crazy to work for them in the creation and validation of standardized tests... do you know how much is an MMPI (Minnesota Multiphasic Personality Inventory) or WAIS (Weschler Adult Intelligence Scale) worth? they're on the 1000s of dollars! and what's weird to me is that those people's primary recruitment pool is the program where you're doing your Master's! i think you mentioned you're in measurement/stats program in the dept of education, rite? (are you in FSU, btw? just me being curious) well, just keep your eyes open for the next career day and you're bound to see them.... they come up north here to UBC every year scouting for talent, i'm sure they scout even more often down there in the States....
 

noetsi

No cake for spunky
#57
I'm assuming you're not counting "Dason: agriculture" under your list of those you consider for private sector business.
Your right since you previously noted you worked for a department of agriculture :)

I am sure spunky that the number of statisticians are 1) very rarely employed at firms as a percent of the total firms (actuerial analyst are likely employed far more commonly) and 2) senior managers at the vast majority of private firms routinely ignore statistical analyst.

One can always point out a small number of firms that do employ statisticians for highly specialized purposes like psychometric tests, analysis of crop production, medicine etc. My point is how common they are and how much influence they have generally. Respectfully we will have to disagree on this point.
 

Link

Ninja say what!?!
#58
I'm a little surprised how this topic shifted from statistical models to generalization (or specialization) of the statistics profession. Just saying...
 

Dason

Ambassador to the humans
#59
Indeed. I probably should have done a better job moderating and moved that type of discussion to a separate thread - but I got caught up in it too! Anywho, from here on out this thread will be used for the original topic and if anybody wants to continue the discussion then feel free to start a separate thread for that.
 

noetsi

No cake for spunky
#60
To restate my original point I think logistics has an advantage because the major commerical softwares print out a odds ratio for Logistic regression but not probit.