joint probability with conditional

xag

New Member
#1
Hello,
What is P(A|B, C)?
I have a dataset and I want to estimate the mutual information of A|B and C.
The formula for mutual information is
Sum_A|B Sum_C p(a|b, c) log( p(a|b, c) / p(a|b) p(b) ) da db
I'm looking for a formula for p(a|b, c)

Many thanks in advance
Xavier
 

xag

New Member
#2
To be more specific, I was looking for a formula for P(D, C) with D = A|B.
The idea being that I want to know how much additional information C brings about A when we already know B.
But I figured that if "we know B for A", "we know B, period" (assuming it makes any sense).
So I'm really looking for the mutual information of A and C knowing B.

P(D, C) = P(A, C | B)

p(a, c | b) = p(a, c, b) / p(b)

I(A, C | B) = Sum_B Sum_A Sum_C p(a, c | b) log( p(a, c | b) / (p(a | b) p(c | b)) )
= Sum_B Sum_A Sum_C (p(a, c, b) / p(b)) log( p(a, c, b) p(b) / (p(a, b) p(c, b)) )

For which a simple (biased) estimator would be
Iest(A, C | B) = Sum_B Sum_A Sum_C (#(a, c, b) / #b) log( #(a, c, b) #b / (#(a, b) #(c, b)) )
where #(x1, ..., xn) is the number of joint occurrences of (x1, ..., xn).

Is this correct?

Thanks,
X.