+ Reply to Thread
Results 1 to 2 of 2

Thread: Interpret GLM model in test of difference in proportions

  1. #1
    Points: 2,395, Level: 29
    Level completed: 64%, Points required for next Level: 55

    Posts
    12
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Interpret GLM model in test of difference in proportions




    I'm facing the following situation: We conducted a marketing campaign with three groups: 2 test vs 1 of control, where two different type of coupons sent to the two test groups, 3 euros vs. 6 euros. I understand that the best way to test the difference in proportion is a GLM model with binomial distribution.

    The dataset is composed of two columns, one referring to the reference group and the other to the redemption result (1=yes, 0=no). The column "enviados" is just a counter, being "1" for all the observations.
    Code: 
    > head(exp11)
      TRAN_DURING_CAMP_FLG enviados bono_recibido
    1                    0        1    BONO6EUROS
    2                    0        1    BONO6EUROS
    3                    0        1    BONO6EUROS
    4                    0        1    BONO6EUROS
    5                    0        1    BONO6EUROS
    6                    0        1    BONO6EUROS
    I wanted to test if there was a significant difference in the redemption rate for each group. Here are the rates:

    Code: 
    > head(testexp)
      bono_recibido TRAN_DURING_CAMP_FLG enviados      ratio
    1     benchmark                 1888     9600 0.19666667
    2    BONO3EUROS                  113     1316 0.08586626
    3    BONO6EUROS                 5449    27227 0.20013222
    Indeed there is a difference in proportion redeemed between the different groups ("ratio" column). To test the significance in difference I applied a GLM model, computed the odds ratios, CIs and ran a post-hoc test:
    
    Call:
    glm(formula = TRAN_DURING_CAMP_FLG ~ bono_recibido, family = "binomial", data = exp2)
    
    Deviance Residuals: 
        Min       1Q   Median       3Q      Max  
    -0.6683  -0.6683  -0.6683  -0.6618   2.2158  
    
    Coefficients:
                            Estimate Std. Error z value Pr(>|z|)    
    (Intercept)             -1.40726    0.02568 -54.805   <2e-16 ***
    bono_recibidoBONO3EUROS -0.95793    0.10166  -9.423   <2e-16 ***
    bono_recibidoBONO6EUROS  0.02179    0.02981   0.731    0.465    
    ---
    Signif. codes:  0 *** 0.001 ** 0.01 * 0.05 . 0.1   1
    
    (Dispersion parameter for binomial family taken to be 1)
    
        Null deviance: 37673  on 38142  degrees of freedom
    Residual deviance: 37548  on 38140  degrees of freedom
    AIC: 37554
    
    Number of Fisher Scoring iterations: 4
    
    > exp(coef(mod.bin)) #odds ratio
                (Intercept) bono_recibidoBONO3EUROS bono_recibidoBONO6EUROS 
                  0.2448133               0.3836877               1.0220305 
    > exp(confint(mod.bin)) # IC
    Waiting for profiling to be done...
                                2.5 %    97.5 %
    (Intercept)             0.2327371 0.2573840
    bono_recibidoBONO3EUROS 0.3127931 0.4661529
    bono_recibidoBONO6EUROS 0.9641824 1.0837123
    
    > summary(glht(mod.bin, mcp(bono_recibido="Tukey")))
    
         Simultaneous Tests for General Linear Hypotheses
    
    Multiple Comparisons of Means: Tukey Contrasts
    
    
    Fit: glm(formula = cbind(TRAN_DURING_CAMP_FLG, enviados - TRAN_DURING_CAMP_FLG) ~ 
        bono_recibido, family = binomial, data = testexp)
    
    Linear Hypotheses:
                                 Estimate Std. Error z value Pr(>|z|)    
    BONO3EUROS - benchmark == 0  -0.95793    0.10169  -9.420   <1e-04 ***
    BONO6EUROS - benchmark == 0   0.02179    0.02981   0.731    0.728    
    BONO6EUROS - BONO3EUROS == 0  0.97972    0.09955   9.841   <1e-04 ***
    ---
    Signif. codes:  0 *** 0.001 ** 0.01 * 0.05 . 0.1   1
    (Adjusted p values reported -- single-step method)
    I have some doubts about how to interpret the results of the test:

    Looking at the GLM model, it seems that the stimulus bono6euro coefficient is not significant, while the stimulus bono3euro and the control group are.

    Question: Do I assume that the fact of having received a stimulus of 3 euro significantly impacts the chance of having the customer redeem the coupon? (In this case negatively as the odds ratio of the 3 euro stimulus is less than the one of the intercept: 0.2448133*0.3836877=0.0939) while the fact of having received a stimulus of 6 euro does not significantly affect the odds ratio relative to not having received any kind of stimulus?
    Looking at the deviances, the residual deviance is slightly lower than the null deviance. I understand it means that the model does not explain much of the variability in the dataset (lower than 1%).

    Question: Can I assume correctly that the fact of having received the stimulus have a low explanatory power of the redemption behaviour in the sample? Which means that the fact of having received one of the two stimuli has not triggered in the test groups any greater spending behaviour than the control group, meaning the stimuli have not worked properly?
    The confidence intervals for the odds ratios do not include "0" and are narrow.

    Question: Can I assume that the odds ratio are then precise enough in telling me that, being <1, the exposure to the stimulus is associated with lower odds of stimulating a purchasing behaviour in the sample? Which means, again, that the stimulus used have not affected the consumer behaviour?

  2. #2
    Points: 8,120, Level: 60
    Level completed: 85%, Points required for next Level: 30

    Posts
    169
    Thanks
    1
    Thanked 7 Times in 7 Posts

    Re: Interpret GLM model in test of difference in proportions


    Maybe your post will become more clear, if you
    1. st present the experiment and your working hyptheses,
    2. nd present present the data
    3. rd present the model building y ~ x1 + x2 +x ...
    4. th present the model results like summary() ...
    5. th ask the questions
    Consuli
    Prediction is very difficult, especially about the future. (Niels Bohr)

+ Reply to Thread

           




Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats