Disagreement among statisticans

[CB]

4) Did I mention that people are idiots? Also some people learn from idiots and then pawn off the wrong facts as true and might even mangle them even more in the process.

It is striking to see how statistics is used by different fields and the strong disagreement by field. Statistics is unusual in that it is broadly used in research and practice by individuals who are not trained statisticians

This is probably the crux of it. There are lots of seemingly respectable books and journal articles about statistics that are written for non-statisticians, by non-statisticians, reviewed by non-statisticians...

And (in part) because actual statisticians are unlikely to be involved in producing or even reading these sources, they can be an echo chamber of bogus claims (like regression requiring normally distributed data, or Kruskal-Wallis being a test of differences in medians, or a that a p value is the probability that the result "was due to chance", etc etc etc).

 

[noetsi]

Yeah Cowboybear. Until I came here most of the statistical text I read were by psychologist, poly sci researchers (in both administration and international relations), engineers, economics, etc. Very rarely did I get a book by a statistician. And when I did, it was often incomprehensible to me because either 1) the wording involved concepts that I was unfamiliar with and the author never defined or 2) the emphasis was in pure math with little actual wording. I have gone through books in time series where whole pages were filled with matrix algebra and or calculus with very little explanation of what they meant. And the author never really built from the math to explain his points.

Once after several pages of this an author noted: "As should be obvious by now..." needless to say it was not at all obvious to me.

This is no different than any other field of course. Each field has its own language which it assumes everyone knows. And journals, and to some extent books as well, have limited space so their is no way to define the terminology or nomeclature even if such was desired.

This reflects a basic criticism I have of all academic fields. Authors are writing for maybe 200 people in the world and really don't care if anyone outside that circle understands what they are doing. It is why academics has, IMHO, very little influence when it should. Because they, it used to include me, are writing for each other only. If academics want research to have real impact on policy, as they should, then they are going to have to bridge the world between other academics and the public (or at the policy making public).

But I doubt from my own experiences most have any interest in doing so.

 

[CB]

Very rarely did I get a book by a statistician. And when I did, it was often incomprehensible to me because either 1) the wording involved concepts that I was unfamiliar with and the author never defined or 2) the emphasis was in pure math with little actual wording.

I hear you. I think there needs to be more collaboration between statisticians and those writing stats books for specific fields. I.e., if a psychologist wants to write a book about stats for psychologists, they should get a statistician to co-author. The statistician makes sure the book makes claims that are correct. The psychologist makes sure the book is comprehensible to psychologists.

This reflects a basic criticism I have of all academic fields. Authors are writing for maybe 200 people in the world and really don't care if anyone outside that circle understands what they are doing.

I wouldn't say we don't care. Sometimes an author would like, in an ideal world, for everyone to understand their findings - but if they're operating at the cutting edge of an advanced field like theoretical physics, that just isn't possible. Sometimes content requires a lot of background knowledge to understand, no matter how nicely it's written! At other times it could be feasible to communicate the knowledge to a broad audience, but since it's people with specialised knowledge in the area who decide whether it'll be published (and therefore whether we have bread on the table), it's them that we have to keep happy.

 

[spunky]

It is why academics has, IMHO, very little influence when it should. Because they, it used to include me, are writing for each other only. If academics want research to have real impact on policy, as they should, then they are going to have to bridge the world between other academics and the public (or at the policy making public).

But I doubt from my own experiences most have any interest in doing so.

well, i think that's debatable. i mean, don't you think big pharma companies hire the best of the best statisticians, biochemists, computer engineers, etc? what more influence do you want to have than being in the company that will find the cure for cancer or something? or when you're a big data analyst in Google and you basically get to decide what shows up on people's google searches? that's top-notch influence, if you ask me.

but when it comes to stepping outside the ivory tower (particularly in mathematics) that can be... well... for some people it's barely on their radar. when i think about academics wanting to have 'influence' in the outside world, it reminds me of a quote attributed to Hardy (but not confirmed):

Here's to pure mathematics—may it never be of any use to anybody.

let's get real here. having any impact in policy has more to do with your politics than your credentials. anyone who's got 2 cents worth of a brain probably knows that. now, if you're aware of this and still expect to have any real influence in the outside world (without playing politics) well... we'll need to call the looney bin so they can come pick you up after a few years in graduate school and meeting newly-minted PhDs i kinda know what everybody does want: a job with a stable paycheque. social idealism usually dies after you get a mortgage

 

[noetsi]

I would actually guess, it is only that of course, the best statisticians, biochemist etc are in universities not corporations or government. Because I think they don't care that much about money - they like the challenge of research and corporations rarely provide that. Corporations are interested in far more narrow, quick results that bore the cutting edge people (or in the case of deadlines greatly annoy them. The time window for academics and business seems vastly different to me).

I don't think corporations do much cutting edge research in honesty. The era of Bell Labs is long past and the narrow focus on making a profit that has dominated US industry in recent decades makes it very unlikely it will return. Corporations make money by guessin a market niche or narrow technical changes commonly. Not breakthroughs which take decades to bring to fruition. Maybe medicine is different, I have my doubts.

While politics has a key role in policy, I think much of policy in fact builds on what is accepted knowledge. And that in fact policy changes when policy makers and the public (at least elites) comes to believe something works. That is one reason that political groups in the US have created think tanks such as Cato, Brookings etc. If academics can show that policy X leads to better results than often it has a chance to be pursued. Certainly I have seen that in my own research.

Amusingly we are now discussing what my research was actually on, organizational behavior and policy change, when I was last an academic researcher. Not the stuff I have done the last decade at all.

 

[spunky]

I would actually guess, it is only that of course, the best statisticians, biochemist etc are in universities not corporations or government. Because I think they don't care that much about money - they like the challenge of research and corporations rarely provide that. Corporations are interested in far more narrow, quick results that bore the cutting edge people (or in the case of deadlines greatly annoy them. The time window for academics and business seems vastly different to me).

I don't think corporations do much cutting edge research in honesty. The era of Bell Labs is long past and the narrow focus on making a profit that has dominated US industry in recent decades makes it very unlikely it will return. Corporations make money by guessin a market niche or narrow technical changes commonly. Not breakthroughs which take decades to bring to fruition. Maybe medicine is different, I have my doubts.

the era of Bell Labs is far from over noetsi. if anything it is growing at a rate so fast that those who are not experts in each sub-field miss it. take Google X, for instance dedicated purely to the advancement of technology. and although, yes, Google expects that whatever comes out from Google X will be something they can eventually sell, the scientists that work there have A LOT of freedom in terms of what they wish to pursue. or the Statistics and Psychometrics division of research at ETS. all the cutting edge statistical developments and the technology to implement them that came out of it is the main reason of why standardized testing looks the way it looks now. from my friends who have done their internships there (and which i will be doing this coming summer) it looks more like someone telling you "hey, look at this really cool problem. wanna work with us in solving it?". and if you don't like that problem, then you look for another one that catches your interest. but if got work on some standard marketing research firm (or something like that) then yes, i would expect that my creativity would be trumped and my curiosity killed for the sake of having the reports ready by monday at 5pm. but that's got more to do with what i'm willing to do and where i'm willing to work. my skills allow me to go to ETS, so i'll go to ETS so they get to sell more tests and i get to play around with psychometric models all day. my skills also allow me to go work for this nameless marketing research firm but i need to be aware of the sacrifices i would make if i go there.

i think you cast a very wide net (as you usually do) when you say "oh, corporations don't do this or don't do that" when, in fact, there is plenty of opportunity to be a happy camper researcher while working in the private sector.

While politics has a key role in policy, I think much of policy in fact builds on what is accepted knowledge. And that in fact policy changes when policy makers and the public (at least elites) comes to believe something works. That is one reason that political groups in the US have created think tanks such as Cato, Brookings etc. If academics can show that policy X leads to better results than often it has a chance to be pursued. Certainly I have seen that in my own research.

when you say "the public (at least elites) comes to believe something works" then question i ask is something works, but for who? or what are these "better results" you're also talking about? who do they benefit?

if the President comes out with a report saying something like "X works" the opposition parties will come with their own report saying "X is bad, Y works better". if we're talking policy-development, academia simply becomes another political arm of the people in power. if you're a politician i bet you'll be able to find countless of research institutes willing to say whatever you want them to say with the necessary data to back it up. if the President says something like "unemployment has been reduced, here are the numbers" the opposition will say "he's using a flawed definition of unemployment, with this new definition look at how bad things are". or "the question is not about unemployment, we need to talk about UNDERemployment as well". who is correct? well, depends on who you want to believe. but as an academic who gets into politics and wants to make a mark, you'll believe whoever is paying you. heck, sometimes you don't even have to be into politics and will still bend the truth in whichever way you see fit.