# How much Linear Algebra is necessary?

#### hedgie

##### New Member
I'm looking to do a PhD in Statistics and and wanted to know everyone's thoughts on how much linear algebra is necessary? I know one needs the first course on linear algebra which is mostly computational but how helpful/necessary is a second proof based course out of Friedberg, Insel, Spence or something? Is that very helpful/necessary?

##### Ninja say what!?!
A lot. You should have covered it in your masters if not undergrad. All the advanced stat stuff relies on it pretty heavily.

#### hedgie

##### New Member
Thanks. I have taken linear algebra. So advanced stat stuff relies heavily on the proof based linear algebra course material?

#### noetsi

##### No cake for spunky
I dont know about proofs, but it relies heavily on the manipulation of matrices. There is also a good deal of algebraic manipulation and some calculus (for example the use of derivatives in maximum liklihood).

#### Dason

I dont know about proofs, but it relies heavily on the manipulation of matrices.
If you get into the theory of linear models then you can get VERY heavy with linear algebra proofs.

*Quitely continues his theory of linear models homework...*

#### hedgie

##### New Member
Haha Dason love that last small line regarding your homework. Thank you all! So consensus is take the proof based more abstract course in addition to the computational course...if one were going to a more applied program would that advice still hold?

#### spunky

##### Can't make spagetti
if one were going to a more applied program would that advice still hold?
i think it depends on what you mean by "applied" and "statistics"... if you mean you're more into a program like quantitative methods or measurement in the social sciences then i think you can get away without the proof-based course... but if by statistics you mean a program housed in the department of math/stats then yes, i think at the PhD level they would expect you to know both the (advanced) theory and applications of linear algebra... heck, even when i toyed around with becoming an actuary they expected me to know stuff about groups and rings... (which is sorta weird but meh...)

#### hedgie

##### New Member
Thanks Spunky. Groups and rings for an actuary...crazy would have never thought that. I was referencing applied stats in stats/business or stats/math dept. Thanks to everyone!

#### noetsi

##### No cake for spunky
Haha Dason love that last small line regarding your homework. Thank you all! So consensus is take the proof based more abstract course in addition to the computational course...if one were going to a more applied program would that advice still hold?
I am currently in a Master's in a (very applied) Measurement and Statistics program. I am generally familiar with the doctoral program for that. It is very light on matrices (which few data analyst actually know I suspect regardless of what statistics they utilize).

So I would say no. Computers do it for you these days.....only at the cutting edge (or the esoteric stuff Dason does) do you use linear algebra calculas etc.

#### Dason

So I would say no. Computers do it for you these days.....only at the cutting edge (or the esoteric stuff Dason does) do you use linear algebra calculas etc.
Which part of what I do is esoteric again?

I would also add that you don't need the linear algebra just for the cutting edge stuff - it's legitimately useful to know to understand what is going on in linear models.

#### hedgie

##### New Member
I am currently in a Master's in a (very applied) Measurement and Statistics program. I am generally familiar with the doctoral program for that. It is very light on matrices (which few data analyst actually know I suspect regardless of what statistics they utilize).

So I would say no. Computers do it for you these days.....only at the cutting edge (or the esoteric stuff Dason does) do you use linear algebra calculas etc.
Aren't you just referring to the computational aspects here? I was asking about the proof based course after the computational course. I already took Linear Algebra which is more the course that is more just matrix manipulations with some light theory.

#### spunky

##### Can't make spagetti
I already took Linear Algebra which is more the course that is more just matrix manipulations with some light theory.
well, then i think you have a pretty neat advantage right there because, at least in the beginning, everything's gonna look very familiar to you... so go ahead and take it! you'll have fun!

#### noetsi

##### No cake for spunky
Aren't you just referring to the computational aspects here? I was asking about the proof based course after the computational course. I already took Linear Algebra which is more the course that is more just matrix manipulations with some light theory.
I am not sure what you mean by this. I meant that you do not need matrix algebra at all for that applied master's (and possibly doctoral) program I was referring too. If you mean is the program I was describing geared to practical applications and not theory then absolutely. It is a highly applied program. Running data, no theory.

#### noetsi

##### No cake for spunky
Which part of what I do is esoteric again?
From this lowly analyst's perspective - everything Do you ever discuss matrix algebra with your boss at work? I amuse myself thinking of my doing that....

#### hedgie

##### New Member
Sorry what I meant is aren't you referring to the computers performing the calculations on the matrices and do the PhD's still not need to interpret and make inferences from this where the higher linear algebra would be helpful. Also just to clarify is this an applied stats program? Thanks noetsi!

#### Dason

It is a highly applied program. Running data, no theory.
The thought of no theory makes me sick. I know not everybody enjoys it but shouldn't you at least know at least a somewhat theoretical basis for what you're doing? I mean if you run across something you've never seen then if you don't know any theory do you just throw your hands up and say "I don't know what to do"?

From this lowly analyst's perspective - everything Do you ever discuss matrix algebra with your boss at work? I amuse myself thinking of my doing that....
I just ask because I feel like I haven't dived too deep into the esoteric realm so far. Heck even what I talk about on the forums I don't feel is too esoteric. I know GLMs aren't as well known but I wouldn't classify them as esoteric - and to be fair GLMs are probably one of the most advanced methods I talk about here.

But in what I do now it's pretty much typical data analysis. Sure we use a few advanced techniques since we're working with gene expression data so we'd be fools to not take advantage of some information sharing. But literally the last thing I did for my RA was run a t-test (albeit I ran 50,000 t-tests...). It doesn't get much more basic than that.

#### spunky

##### Can't make spagetti
I mean if you run across something you've never seen then if you don't know any theory do you just throw your hands up and say "I don't know what to do"?
that is exactly what people do Dason. as someone who made the transition from mathematics to quantitative methodology on the social sciences, i am part of a small-but-growing movement (which counts some of the most influential names in our field like peter bentler or denny borsboom) that constantly decries the way statistics is taught both at the undergraduate and graduate level programs in education, psychology, political science, sociology or any other social science that uses quantitative research methods. and the first problem on the list is exactly that: not enough emphasis on the theoretical underpinnings of the methods used. in my program of study (and many, many others) what you will get at most are a few handwaving arguments regarding how things work, a couple of in-class examples and a dataset to practice at home. the emphasis is on (a) how to enter the data on the software and (b) how to read the tables the software spews back, following a cookbook-like format. oh, and if SPSS can't do it,then forget it... any alternative kind of analysis that is either not pre-built on SPSS or that can be done from information provided by SPSS is unused and labelled as "too complicated" or "for experts only".

and yes, i have to rant about this because the moment people run out of the standard tricks they're taught (like fixing multicollinearity by substracting the means, eigenvalues-greater-than-one to retain factors in factor analysis, using bonferroni or sidak corrections of type 1 error rate, etc.) in the middle of the semester, right before the deadline for grant applications and thesis proposals you start getting this endless line of people outside the office of the statistical consultant (aka the dept data-analysis-slave $$\triangleq$$ me) asking how to do this or work out that.

for a much more comprehensive and fun version of this rant, i always recommend Denny Borsboom's The Attack of the Psychometricians

#### BGM

##### TS Contributor

Recently the statistical research field seems to have many hot topics about the high-dimensional data analysis, in which a good linear algebra knowledge is a must.

For example if you started to learn about the principle component analysis, you need to have the basic knowledge about the eigenvalue; if you do not know linear algebra well, you may even have a hard time to understand the term "degrees of freedom" as posted in the other thread.

And when you try to read the other researcher's statistical paper, you can get the feeling of the requirement of your linear algebra knowledge.

At last I want to share one of my professor advises: If we study the elementary calculus and linear algebra courses well, then we can build up a much better mathematical foundation to prepare for the more difficult courses in the future, with most of the concepts are developed from these two courses.

#### noetsi

##### No cake for spunky
The thought of no theory makes me sick.
Which is one reason I am in awe of you! Only one of course. But I think you would find that for 99.99 percent of the data analyst in the private sector the idea of being concerned with theory is entirely alien to their make up or job.

But in what I do now it's pretty much typical data analysis. Sure we use a few advanced techniques since we're working with gene expression data so we'd be fools to not take advantage of some information sharing. But literally the last thing I did for my RA was run a t-test (albeit I ran 50,000 t-tests...). It doesn't get much more basic than that.
Try chi square and descriptive statistics thrown into excel tables. And being considered "cutting edge" for that.... In the, relatively sophisticated by private sector firm I worked at the CEO greeted one analysis by asking, "which number is larger."