# Distribution of MLE

#### Darktranquillity

##### New Member
Hey guys how are you? I have the following question:

Let X1,X2,...,Xn be a random sample from a Pareto distribution having pdf
f(x|b)= (a*b^a)/x^(a+1) where x>=b (1)

Determine the maximum likelihood estimator for b, say b' on (0,infinity) and by considering P(b'>x) or otherwise show that b' has the Pareto distribution with pdf given by (1) but with a replaced by an.

My attempt: I found the MLE as b'=min Xi where 1<=i<=n, since our pdf is monotonically increasing w.r.t b.

After that I know how to find the asymptotic distribution of the MLE using the formula including the expected information but then we say that MLE follows a normal distribution for large n.

How do I show that the MLE follows a Pareto distribution in this case? I am so struggled, any help would be much appreciated!

P.S The hint tells us to consider P(b'>x) but how can I find P(min Xi >x) and why should it help me?

#### Dason

P.S The hint tells us to consider P(b'>x) but how can I find P(min Xi >x) and why should it help me?
It's a hint on deriving the distribution of the sample minimum (which in this case is the distribution of your mle).

Edit: If you're looking for the interesting off-topic discussion that was contained in this thread before then you might want to turn your attention... here.

#### Darktranquillity

##### New Member
The sample minimum is every time the smallest of our observations. Where does this lead me? Consider also that x>=b for the previous pdf, so the probability of b' to be >x wouldn't be 0, using that x>=b??

Really I am so struggled. Can you please help me more? I can't think anything else at all!

Last edited:

#### Dason

If you wanted to know P(X(min) >= b) this is the same as P(X1 >= b, X2 >= b, X3 >= b, ..., Xn >= b) right? So use independence and a little bit of math to see if you can get the distribution from that.

#### Darktranquillity

##### New Member
If you wanted to know P(X(min) >= b) this is the same as P(X1 >= b, X2 >= b, X3 >= b, ..., Xn >= b) right? So use independence and a little bit of math to see if you can get the distribution from that.
P(min Xi >x) is the same as P(min Xi >= b), since x>=b? But then why P(minXi >= b) is the same as P(X1>=b, X2>=b,.....)? I mean onle one is the sample minimun, why do we use every Xn?
Is it possible to provide me the solution since there have been to many days I am trying to find a solution? Thank you very much, I hope I am not asking for too much!

Last edited:

#### Dason

If b < Min of Xi then b has to be less than EVERY one of the Xi. So if we're looking at the cdf of X(min) we want to know P(X(min) <= b) = 1 - P(X(min) > b) = 1 - P(X1 > b, X2 > b, X3 > b, ..., Xn > b)

Can you see where to go from here?

#### Darktranquillity

##### New Member
If b < Min of Xi then b has to be less than EVERY one of the Xi. So if we're looking at the cdf of X(min) we want to know P(X(min) <= b) = 1 - P(X(min) > b) = 1 - P(X1 > b, X2 > b, X3 > b, ..., Xn > b)

Can you see where to go from here?
So if I understand well: We want to find the pdf of x(min) = b'. But we know that b<=x so we need b<=min(x). The hint of the question tells me to consider P(b'>x) i.e
P(xmin>x). But you tell me to look at P(xmin<=b) not P(xmin>x) that's what I am confused about...

#### Dason

It doesn't matter. P(xmin <= b) = 1 - P(xmin > b)... so it's the same thing as the hint really...

#### Darktranquillity

##### New Member
But we know that for a Paretto distribution any x is >= b. So P(x>=b) = 1, since is always true if we are talking for a Paretto distribution. So P(xmin>=b) = 1, since xmin is any of the xn. What am I thinking incorrectly?!

#### BGM

##### TS Contributor
But we know that for a Paretto distribution any x is >= b. So P(x>=b) = 1, since is always true if we are talking for a Paretto distribution. So P(xmin>=b) = 1
Yes you are correct, but back to the hints in the first post:

P.S The hint tells us to consider P(b'>x)
So now you know how to calculate this? for a general $$x > b$$.

(you already know when $$x \leq b$$ the probability is 1)

#### Darktranquillity

##### New Member
Yes you are correct, but back to the hints in the first post:

So now you know how to calculate this? for a general $$x > b$$.

(you already know when $$x \leq b$$ the probability is 1)
But a Pareto distribution needs x>=b so P(x<b)=0 [for any x] so P(xmin<b)=0 since we are talking for a Pareto distribution! But P(x>b)=1 and P(xmin>b) or P(b'>b)=1. How do I find P(xmin>x) or P(b'>x) where b' = xmin?

#### BGM

##### TS Contributor
First of all try to think about the following relationship :

1. If $$\min_i x_i > x$$, then $$x_1 > x, x_2 > x, ..., x_n > x$$

2. If $$x_1 > x, x_2 > x, ..., x_n > x$$, then $$\min_i x_i > x$$

i.e. The two events are equivalent (if and only if).

And thus the probability of the corresponding events are equal.

#### Dason

Yeah I got sloppy with notation and was just trying to get you to think about P(Xmin > b) for any constant b (not necessarily the b that is in the pareto distribution).

#### Darktranquillity

##### New Member
First of all try to think about the following relationship :

1. If $$\min_i x_i > x$$, then $$x_1 > x, x_2 > x, ..., x_n > x$$

2. If $$x_1 > x, x_2 > x, ..., x_n > x$$, then $$\min_i x_i > x$$

i.e. The two events are equivalent (if and only if).

And thus the probability of the corresponding events are equal.
So I have to find P(x1>x, x2>x,....,xn>x) to find the CDF of the sample minimum and then integrate it to find its pdf?? If this is the way how do I find P(x1>x, x2>x,....,xn>x) ?

#### BGM

##### TS Contributor
Ok assuming you agree on that fact. Several things to point out:

1. $$\Pr\left\{\min_{i=1,2,...,n}X_i > x\right\}$$ is not the CDF; it is the survival function. $$\Pr\left\{\min_{i=1,2,...,n}X_i \leq x\right\}$$ is the CDF.

2. Usually in these problems, the random sample $$X_1, X_2, ..., X_n$$ are independent unless otherwise specified.

#### Darktranquillity

##### New Member
Ok assuming you agree on that fact. Several things to point out:

1. $$\Pr\left\{\min_{i=1,2,...,n}X_i > x\right\}$$ is not the CDF; it is the survival function. $$\Pr\left\{\min_{i=1,2,...,n}X_i \leq x\right\}$$ is the CDF.

2. Usually in these problems, the random sample $$X_1, X_2, ..., X_n$$ are independent unless otherwise specified.
Ok, so I find the cdf and then integrate to find its pdf? But how do I find the cdf?

#### BGM

##### TS Contributor
By differertiating a CDF you get the pdf.

Also, you know that the random sample are independent - so what hinder you to calculate the CDF???

If this is the way how do I find P(x1>x, x2>x,....,xn>x) ?

#### Darktranquillity

##### New Member
I am really struggled to find the cdf, I really can't recall anything to find it!

#### BGM

##### TS Contributor
At this stage, beside directly telling you the answer, let me sum up the things for you and see you stuck at which step.

1. The MLE $$\hat{b} = \min_{i=1,2,...,n}X_i$$ (already found by you)

2. To show that the MLE has a certain distribution, we consider the survival function
$$\Pr\{\hat{b} > x\}$$ (equivalent to consider the CDF)

3. By the definition of minimum,
$$\Pr\left\{\min_{i=1,2,...,n}X_i > x\right\} = \Pr\{X_1 > x, X_2 > x, ..., X_n > x\}$$

4. Since the random sample $$X_1, X_2, ..., X_n$$ are independent and identically distributed as a Pareto distribution, the above survival function become ...

5. Finally can you figure out the resulting survival function is the survival function of a Pareto distribution? and what is the corresponding parameters?

#### Darktranquillity

##### New Member
At this stage, beside directly telling you the answer, let me sum up the things for you and see you stuck at which step.

1. The MLE $$\hat{b} = \min_{i=1,2,...,n}X_i$$ (already found by you)

2. To show that the MLE has a certain distribution, we consider the survival function
$$\Pr\{\hat{b} > x\}$$ (equivalent to consider the CDF)

3. By the definition of minimum,
$$\Pr\left\{\min_{i=1,2,...,n}X_i > x\right\} = \Pr\{X_1 > x, X_2 > x, ..., X_n > x\}$$

4. Since the random sample $$X_1, X_2, ..., X_n$$ are independent and identically distributed as a Pareto distribution, the above survival function become ...

5. Finally can you figure out the resulting survival function is the survival function of a Pareto distribution? and what is the corresponding parameters?
Ok, since they are independent we can say that P(X1>x, X2>x,...,Xn>x)= P(X1>x) * P(X2>x) * ....*P(Xn>x)= [P(X1>x)]^n?? But then what is P(X1>x)? Have I used correctly the independence?