Maximizing expection by selecting from 2 binomial processes

[mlut]

Please help!

I have two unfair coins with unknown p's. I am allowed to toss a total of N times, and the problem is to devise a strategy which will yield the maximum number of heads over these N tosses.

My instinct says that in order to decide before each toss, I will need to figure out the distributions of the estimates of p1 and p2, and use their overlap in some way. Not really sure how to develop this further.

Any ideas?
Fiorella

[AtlasFrysmith]

Cool problem. I think the idea is to flip whichever coin you think at that time has the higher p, so for the first flip it doesn't matter, the second you flip again if it's heads or switch if not, etc, always adjusting your posterior estimates of the probabilities in light of earlier flips. But I'm not sure it's so simple as just taking the point estimate of each probability. For example, that oculd leave you with the following: If you flip coin A once and get tails, your posterior probability is , so you'd flip coin B next, and if you got heads you'd flip B forever, no matter what else happened. But suppose after a while you get an estimate that's really low. Then at some point you'd have to wonder if your one-time estimate on coin A was just a fluke and it's worth trying again. So it seems that what you really want to calculate at each stage is . So you'd need the joint distribution .

That's possible to compute, though a bit more difficult. What worries me, and I don't have time or possibly the knowledge to work out, is if there are cases in which the information value of flipping the worse but more uncertain coin outweighs the expected loss, even when we're considering the joint probabilities. I'm quite tired right now and have some of my own work to do, but this is pretty interesting to me and I may come back to it. In the meantime, I'd work on deriving the joint conditional distribution above.

PS The fact that N is finite may also play into this; it might work out that the information value works out in such a way that some experiement of X flips makes sense to run at the beginning but not near the end because you'd run out of time. Knowing that, you'd have to adjust what your plans are earlier on too.

[mlut]

You're right - N does matter, e.g, after N-1 tosses you'd obviously just pick the coin with the higher point estimate.
I wonder if the converse is true as well, i.e. that if N is infinity then you would always flip the more uncertain coin?

[fed1]

This is fairly interesting question.

I believe it is tied to a topic called 'sequential testing'. If you get ASA newsletter then you probably got the Data Guide, that has an article on this topic.

It will make big difference whether you make it so the rule for deciding what coin to flip has to be decided before the experiment, or if the rule for deciding what coin to flip can change based on the observed data.