To be more specific, I was looking for a formula for P(D, C) with D = A|B.
The idea being that I want to know how much additional information C brings about A when we already know B.
But I figured that if "we know B for A", "we know B, period" (assuming it makes any sense).
So I'm really looking for the mutual information of A and C knowing B.
P(D, C) = P(A, C | B)
p(a, c | b) = p(a, c, b) / p(b)
I(A, C | B) = Sum_B Sum_A Sum_C p(a, c | b) log( p(a, c | b) / (p(a | b) p(c | b)) )
= Sum_B Sum_A Sum_C (p(a, c, b) / p(b)) log( p(a, c, b) p(b) / (p(a, b) p(c, b)) )
For which a simple (biased) estimator would be
Iest(A, C | B) = Sum_B Sum_A Sum_C (#(a, c, b) / #b) log( #(a, c, b) #b / (#(a, b) #(c, b)) )
where #(x1, ..., xn) is the number of joint occurrences of (x1, ..., xn).
Is this correct?
Thanks,
X.
