# question about likelihood function/MLE

#### messianic

##### New Member
If a random variable Y has the following probability distribution:
P (Y = 1) = p, P (Y = 2) = q, and P (Y = 3) = 1−p−q. A random sample of size
n is drawn from this distribution and the random variables are denoted Y1, Y2, · · · , Yn.

How would you derive the likelihood function for p and q? Would you just sum the probability distributions first?

How would you derive the MLE for p and q?

If someone could get me in the right direction that would be appreciated

#### BGM

##### TS Contributor
Ok. So now you are given

$$\Pr\{Y = 1\} = p, \Pr\{Y = 2\} = q, \Pr\{Y = 3\} = 1 - p - q$$

The probability mass function of $$Y$$ is
$$f_Y(y) = \left\{ \begin{matrix} p & ~ {\rm if} ~ y = 1 \\ q & ~ {\rm if} ~ y = 2 \\ 1 - p - q & ~ {\rm if} ~ y = 3 \\ 0 & ~ {\rm otherwise} \end{matrix} \right.$$

However, you can write a more condensed product form:
$$f_Y(y) = \left\{ \begin{matrix} p^{\frac {(y-2)(y-3)} {2}} q^{-(y-1)(y-3)} (1 - p - q)^{\frac {(y-1)(y-2)} {2}}& ~ {\rm if} ~ y = 1, 2, 3 \\ 0 & ~ {\rm otherwise} \end{matrix} \right.$$

such that the joint log-likelihood for n sample
$$L(p,q;y_1, ..., y_n) = a \ln p + b \ln q + c \ln (1 - p - q)$$

where $$a= \frac {1} {2} \sum_{i = 1}^n (y_i - 1)(y_i - 2), b = -\sum_{i = 1}^n (y_i - 1)(y_i - 3), c = \frac {1} {2} \sum_{i = 1}^n (y_i - 2)(y_i - 3) = n - a - b$$
which can be readily obtained from your sample.

Now we take the partial differentiation with respect to the parameters
and set them equal to 0:
$$\left.\frac {\partial L} {\partial p} \right|_{p = \hat{p}, q = \hat{q}} = \frac {a} {\hat{p}} - \frac {c} {1 - \hat{p} - \hat{q}} = 0$$

$$\left.\frac {\partial L} {\partial q} \right|_{p = \hat{p}, q = \hat{q}} = \frac {b} {\hat{q}} - \frac {c} {1 - \hat{p} - \hat{q}} = 0$$

Simplify them, we have a 2 by 2 linear system in $$\hat{p}, \hat{q}$$:
$$\left\{\begin{matrix} (a+c)\hat{p} + a\hat{q} = a \\ b\hat{p} + (b+c)\hat{q} = b \end{matrix}\right.$$

Eventually, we have
$$\hat{p} = \frac {a} {n}, \hat{q} = \frac {b} {n}$$
Again note $$a + b + c = n$$

But note that a, b, c are just counts of the outcomes 1, 2, 3.
So the MLE are just the ratio of the specific counts to the total sample.

#### messianic

##### New Member
how did you derive the condensed product form?

#### Dason

##### Ambassador to the humans
Well just look what happens to it when you plug in the values y=1, y=2, y=3. You get what you want. Notice that for y = 1 we want it to be p. If we plug in y = 2 we get p^{(2-2)(2-3)/2} = p^0, if we plug in y = 3 we get p^0 again. It's just being clever about how you set up the exponents so that things cancel and give you what you want.

#### messianic

##### New Member
if someone could verify my answer to the MLE for p and q that would be much appreciated,

p = (1-q)/2

q = (1-p)/2

#### BGM

##### TS Contributor
I guess the most important thing here is not the tricky product form.
It just ease my presentation.

In fact, without mentioning that we can also write down the likelihood
function, which is the most important thing you need to know:
the exponents of each outcome in those kind of "multinomial" distribution
are just the observed frequency for each outcome.

PS: Sorry not quite understand the follow up post.
Anyway it is an easy linear system and you can solve it to get
$$p = q = \frac {1} {3}$$

#### Dason

##### Ambassador to the humans
if someone could verify my answer to the MLE for p and q that would be much appreciated,

p = (1-q)/2

q = (1-p)/2
Those don't really look like estimators to me. Since you don't know either p or q to begin with how would you calculate these once you gather data?