So I was still interested in this problem, but for the wrong reasons. I believe this is a GLMM which can be fit by lmer in the lme4 package, that made it interesting to me. But the wrong reasons are I have no experience with GLMMs, lmer or lme4. So that's not helpful.

But I thought about what I would do if someone held a gun to my head.

My model is fixed effects for subgroup and newcriteria levels. And random effects for intercept nested in cycle .

That leads to the model in R of:

promoted ~ subgroup+newcriteria + (1|cycle)

My code was this :

And that leads to summary

Based on that you have statistically significant evidence that subgroup 1 is less likely to be promoted, but you are marginal on evidence that the new criteria is a non-zero fixed effect. This is kind of a more data situation another cycle might just tilt it one way or another.

I am not really sure about diagnostics. Like I said new turf for me. Seek out an expert on GLMMs.

BTW that number of observations line is kind of misleading in this context. Consider:

So now there is 1 observation per a line with summary:

You'll note that even the variance and Std. Dev estimates for the random effect are exactly the same, yet the number of observations has jumped to 17081. It is interesting the likelihood has changed, which suggest internally something different is being done even though the estimates are exactly the same.

edit and ps: According to MASS the first column in the response matrix is considered "success". Which is different than what I said earlier =) And its also confusing because if you ever use the factor version of the response ( column of 1s and 0s) the first factor level is considered failure and everything else is considered success. That's how I messed it up in my head.