Distribution of MLE

#1
Hey guys how are you? I have the following question:

Let X1,X2,...,Xn be a random sample from a Pareto distribution having pdf
f(x|b)= (a*b^a)/x^(a+1) where x>=b (1)

Determine the maximum likelihood estimator for b, say b' on (0,infinity) and by considering P(b'>x) or otherwise show that b' has the Pareto distribution with pdf given by (1) but with a replaced by an.


My attempt: I found the MLE as b'=min Xi where 1<=i<=n, since our pdf is monotonically increasing w.r.t b.

After that I know how to find the asymptotic distribution of the MLE using the formula including the expected information but then we say that MLE follows a normal distribution for large n.

How do I show that the MLE follows a Pareto distribution in this case? I am so struggled, any help would be much appreciated!



P.S The hint tells us to consider P(b'>x) but how can I find P(min Xi >x) and why should it help me?
 

Dason

Ambassador to the humans
#2
P.S The hint tells us to consider P(b'>x) but how can I find P(min Xi >x) and why should it help me?
It's a hint on deriving the distribution of the sample minimum (which in this case is the distribution of your mle).

Think about this - if X is your sample minimum... what can you say about each and every one of the values in your sample? How can you relate this to the hint?

Edit: If you're looking for the interesting off-topic discussion that was contained in this thread before then you might want to turn your attention... here.
 
#3
The sample minimum is every time the smallest of our observations. Where does this lead me? Consider also that x>=b for the previous pdf, so the probability of b' to be >x wouldn't be 0, using that x>=b??

Really I am so struggled. Can you please help me more? I can't think anything else at all!
 
Last edited:

Dason

Ambassador to the humans
#4
If you wanted to know P(X(min) >= b) this is the same as P(X1 >= b, X2 >= b, X3 >= b, ..., Xn >= b) right? So use independence and a little bit of math to see if you can get the distribution from that.
 
#5
If you wanted to know P(X(min) >= b) this is the same as P(X1 >= b, X2 >= b, X3 >= b, ..., Xn >= b) right? So use independence and a little bit of math to see if you can get the distribution from that.
P(min Xi >x) is the same as P(min Xi >= b), since x>=b? But then why P(minXi >= b) is the same as P(X1>=b, X2>=b,.....)? I mean onle one is the sample minimun, why do we use every Xn?
Is it possible to provide me the solution since there have been to many days I am trying to find a solution? Thank you very much, I hope I am not asking for too much!
 
Last edited:

Dason

Ambassador to the humans
#6
If b < Min of Xi then b has to be less than EVERY one of the Xi. So if we're looking at the cdf of X(min) we want to know P(X(min) <= b) = 1 - P(X(min) > b) = 1 - P(X1 > b, X2 > b, X3 > b, ..., Xn > b)

Can you see where to go from here?
 
#7
If b < Min of Xi then b has to be less than EVERY one of the Xi. So if we're looking at the cdf of X(min) we want to know P(X(min) <= b) = 1 - P(X(min) > b) = 1 - P(X1 > b, X2 > b, X3 > b, ..., Xn > b)

Can you see where to go from here?
So if I understand well: We want to find the pdf of x(min) = b'. But we know that b<=x so we need b<=min(x). The hint of the question tells me to consider P(b'>x) i.e
P(xmin>x). But you tell me to look at P(xmin<=b) not P(xmin>x) that's what I am confused about...
 
#9
But we know that for a Paretto distribution any x is >= b. So P(x>=b) = 1, since is always true if we are talking for a Paretto distribution. So P(xmin>=b) = 1, since xmin is any of the xn. What am I thinking incorrectly?!
 

BGM

TS Contributor
#10
But we know that for a Paretto distribution any x is >= b. So P(x>=b) = 1, since is always true if we are talking for a Paretto distribution. So P(xmin>=b) = 1
Yes you are correct, but back to the hints in the first post:

P.S The hint tells us to consider P(b'>x)
So now you know how to calculate this? for a general \( x > b \).

(you already know when \( x \leq b \) the probability is 1)
 
#11
Yes you are correct, but back to the hints in the first post:



So now you know how to calculate this? for a general \( x > b \).

(you already know when \( x \leq b \) the probability is 1)
But a Pareto distribution needs x>=b so P(x<b)=0 [for any x] so P(xmin<b)=0 since we are talking for a Pareto distribution! But P(x>b)=1 and P(xmin>b) or P(b'>b)=1. How do I find P(xmin>x) or P(b'>x) where b' = xmin?
 

BGM

TS Contributor
#12
First of all try to think about the following relationship :

1. If \( \min_i x_i > x \), then \( x_1 > x, x_2 > x, ..., x_n > x \)

2. If \( x_1 > x, x_2 > x, ..., x_n > x \), then \( \min_i x_i > x \)

i.e. The two events are equivalent (if and only if).

And thus the probability of the corresponding events are equal.
 

Dason

Ambassador to the humans
#13
Yeah I got sloppy with notation and was just trying to get you to think about P(Xmin > b) for any constant b (not necessarily the b that is in the pareto distribution).
 
#14
First of all try to think about the following relationship :

1. If \( \min_i x_i > x \), then \( x_1 > x, x_2 > x, ..., x_n > x \)

2. If \( x_1 > x, x_2 > x, ..., x_n > x \), then \( \min_i x_i > x \)

i.e. The two events are equivalent (if and only if).

And thus the probability of the corresponding events are equal.
So I have to find P(x1>x, x2>x,....,xn>x) to find the CDF of the sample minimum and then integrate it to find its pdf?? If this is the way how do I find P(x1>x, x2>x,....,xn>x) ?
 

BGM

TS Contributor
#15
Ok assuming you agree on that fact. Several things to point out:

1. \( \Pr\left\{\min_{i=1,2,...,n}X_i > x\right\} \) is not the CDF; it is the survival function. \( \Pr\left\{\min_{i=1,2,...,n}X_i \leq x\right\} \) is the CDF.

2. Usually in these problems, the random sample \( X_1, X_2, ..., X_n \) are independent unless otherwise specified.
 
#16
Ok assuming you agree on that fact. Several things to point out:

1. \( \Pr\left\{\min_{i=1,2,...,n}X_i > x\right\} \) is not the CDF; it is the survival function. \( \Pr\left\{\min_{i=1,2,...,n}X_i \leq x\right\} \) is the CDF.

2. Usually in these problems, the random sample \( X_1, X_2, ..., X_n \) are independent unless otherwise specified.
Ok, so I find the cdf and then integrate to find its pdf? But how do I find the cdf?
 

BGM

TS Contributor
#17
By differertiating a CDF you get the pdf.

Also, you know that the random sample are independent - so what hinder you to calculate the CDF???

If this is the way how do I find P(x1>x, x2>x,....,xn>x) ?
 

BGM

TS Contributor
#19
At this stage, beside directly telling you the answer, let me sum up the things for you and see you stuck at which step.

1. The MLE \( \hat{b} = \min_{i=1,2,...,n}X_i \) (already found by you)

2. To show that the MLE has a certain distribution, we consider the survival function
\( \Pr\{\hat{b} > x\} \) (equivalent to consider the CDF)

3. By the definition of minimum,
\( \Pr\left\{\min_{i=1,2,...,n}X_i > x\right\} = \Pr\{X_1 > x, X_2 > x, ..., X_n > x\} \)

4. Since the random sample \( X_1, X_2, ..., X_n \) are independent and identically distributed as a Pareto distribution, the above survival function become ...

5. Finally can you figure out the resulting survival function is the survival function of a Pareto distribution? and what is the corresponding parameters?
 
#20
At this stage, beside directly telling you the answer, let me sum up the things for you and see you stuck at which step.

1. The MLE \( \hat{b} = \min_{i=1,2,...,n}X_i \) (already found by you)

2. To show that the MLE has a certain distribution, we consider the survival function
\( \Pr\{\hat{b} > x\} \) (equivalent to consider the CDF)

3. By the definition of minimum,
\( \Pr\left\{\min_{i=1,2,...,n}X_i > x\right\} = \Pr\{X_1 > x, X_2 > x, ..., X_n > x\} \)

4. Since the random sample \( X_1, X_2, ..., X_n \) are independent and identically distributed as a Pareto distribution, the above survival function become ...

5. Finally can you figure out the resulting survival function is the survival function of a Pareto distribution? and what is the corresponding parameters?
Ok, since they are independent we can say that P(X1>x, X2>x,...,Xn>x)= P(X1>x) * P(X2>x) * ....*P(Xn>x)= [P(X1>x)]^n?? But then what is P(X1>x)? Have I used correctly the independence?