PDF of (nearly) collinear variables

[Sam Vimes]

My question is motivated by a problem with estimation by maximum likelihood. In simple terms, I want to estimate a parameter \(\beta \) and have three variables, \(X\), \(Y\) and \(Z\), such that \(Z = X+Y\). Using all three variables is pointless and the joint pdf is singular. Now, suppose that instead of \(Z\) I have \(U\) such that \(U = Z + \epsilon\), where \(\epsilon\) is independent from \(X\) and \(Y\) and its distribution does not depend on \(\beta\). The joint pdf of \(X\), \(Y\) and \(U\) is not singular. Seems to me obvious that \(U\) does not contain any additional information about \(\beta\), given \(X\) and \(Y\). So, I think it must be true that, in terms of the likelihood function, \(L(\beta;X,Y,U)=L(\beta;X,Y)*const\). If I am correct, what would be the best way to prove it? Thanks.

 

[BGM]

The question is equivalent to show that the conditional pdf

\( f_{U|X=x,Y=y}(u|x,y) = \frac {f_{U,X,Y}(u,x,y)} {f_{X,Y}(x,y)} \)

is independent of the parameter \( \beta \)

Consider the conditional CDF

\( F_{U|X=x,Y=y}(u|x,y) \)

\( = \Pr\{U \leq u|X = x, Y = y\} \)

\( = \Pr\{X + Y + \epsilon \leq u |X = x, Y = y\} \)

\( = \Pr\{\epsilon \leq u - x - y|X = x, Y = y\} \)

\( = \Pr\{\epsilon \leq u - x - y\} \) (by independence)

\( = F_\epsilon(u - x - y) \)

By assumption, the distribution of \( \epsilon \) is independent of \( \beta \), and therefore

\( F_{U|X=x,Y=y}(u|x,y) = F_\epsilon(u - x - y)\)

is independent of the beta, and similar for its derivative:

\( f_{U|X=x,Y=y}(u|x,y) = \frac {\partial F_{U|X=x,Y=y}(u|x,y)} {\partial u}\)