a while ago (before Englund became an MVC) I posted a proof about another result in factor analysis. I thought it would be nice to resurrect it (briefly) and add it here to our small (but growing) compendium of proofs. the original thread is here

http://www.talkstats.com/showthread.php/45041-Factor-Analysis-PROOF?highlight=
and the proof goes like this:

Let [MATH]\bf{S}[/MATH] be a covariance matrix with eigenvalue-eigenvector pairs ([MATH]\lambda_1, \mathbf{e}_1[/MATH]), ([MATH]\lambda_2, \mathbf{e}_2[/MATH]), ..., ([MATH]\lambda_p, \mathbf{e}_p[/MATH]), where

[MATH]\lambda_1 \ge \lambda_2 \ge ... \ge \lambda_p[/MATH]. Let [MATH]m<p[/MATH] and define:

[MATH]\bf{L} = \{l_{ij}\} = \left[\sqrt{\lambda_1 }\mathbf{e}_1\ |\ \sqrt{\lambda_2} \mathbf{e}_2\ |\ ...\ |\ \sqrt{\lambda_m} \mathbf{e}_m \right] [/MATH]

and:

[MATH]\

\mathbf\Psi =

\left(

\begin{array}{cccc}

\psi_1 & 0 & ... & 0 \\

0 & \psi_2 & ... & 0 \\

\vdots & \vdots & \ddots & \vdots \\

0 & 0 & ... & \psi_p \\

\end{array}

\right)

\text{ with } \psi_i = s_{ii} - \sum_{j=1}^{m} l_{ij}^2[/MATH]

Then, PROVE:

[MATH]

\text{Sum of squared entries of } (\mathbf{S} - (\mathbf{LL'} + \mathbf{\Psi})) \le \lambda_{m+1}^2 + \cdots + \lambda_p^2[/MATH]

Spunky's attempt of a proof:

By definition of [MATH]\psi_i[/MATH], we know that the diagonal of [MATH](\mathbf{S} - (\mathbf{LL'} + \mathbf{\Psi}))[/MATH] is all zeroes. Since

[MATH](\mathbf{S} - (\mathbf{LL'} + \mathbf{\Psi})))[/MATH] and [MATH](\mathbf{S} - \mathbf{LL'})[/MATH] have the same elements except on the diagonal, we know that

[MATH]\text{(Sum of squared entries of } (\mathbf{S} - (\mathbf{LL'} + \mathbf{\Psi}))) \leq \text{ Sum of squared entries of } (\mathbf{S} - \mathbf{LL'}) [/MATH]

Since [MATH]\mathbf{S} = \lambda_1 \mathbf{e}_1 \mathbf{e}'_1 + \cdots + \lambda_p \mathbf{e}_p \mathbf{e}'_p [/MATH]

and [MATH]\mathbf{LL'} = \lambda_1 \mathbf{e}_1 \mathbf{e}'_1 + \cdots + \lambda_m \mathbf{e}_m \mathbf{e}'_m [/MATH], then it follows that

[MATH]\mathbf{S} - \mathbf{LL'} = \lambda_{m+1} \mathbf{e}_{m+1} \mathbf{e}'_{m+1} + \cdots + \lambda_p \mathbf{e}_p \mathbf{e}'_p[/MATH]

Writing it in matrix form, this is saying [MATH]\mathbf{S} - \mathbf{LL'} = \mathbf{P}_2 \mathbf{\Lambda}_2 \mathbf{P}'_2[/MATH] where

[MATH]\mathbf{P}_2 = [ \mathbf{e}_{m+1} | \cdots | \mathbf{e}_p ][/MATH] and [MATH]\mathbf{\Lambda}_2 = Diag(\lambda_{m+1}, \cdots, \lambda_{p})[/MATH]

Then, the following is true:

[MATH]\text{Sum of squared entries of }(\mathbf{S}- \mathbf{LL'})= \text{tr}((\mathbf{S} - \mathbf{LL'}) (\mathbf{S} - \mathbf{LL'})')=[/MATH]

[MATH]\text{tr} (( \mathbf{P}_2 \mathbf{\Lambda}_2 \mathbf{P}'_2)( \mathbf{P}_2 \mathbf{\Lambda}_2 \mathbf{P}'_2)')=\text{tr}( \mathbf{P}_2 \mathbf{\Lambda}_2\mathbf{\Lambda}_2 \mathbf{P}'_2)[/MATH]

[MATH]tr(\mathbf{\Lambda}_2\mathbf{\Lambda}_2)=\lambda_{m+1}^2 + \cdots + \lambda_p^2.[/MATH]

All the [MATH]\bf{P}_2[/MATH] disappear because by the definition of [MATH]\bf{P}_2[/MATH] we know that [MATH]\bf{P}_2 '\bf{P}_2=\bf{I}[/MATH]