Well this is something I am not good at, so let me provide some advice

This reminds me of the issues of unequal error terms in linear regression. I am trying to remember the remedies. You have these confidence intervals you can construct based on pooled data. So if we picture a 45 degree line and the actual distribution of data shaped like a funnel (say triangle) around it, you risk your smaller point estimates having too broad of confidence intervals and the larger point estimates having too narrow confidence intervals.

Hmm, with a lack of the actual solution (because I am too applied), I wonder if you could do a bootstrap nonparametric confidence interval. So a resampling technique. Let us see if anyone else has input.