Advice on Statistical Test

#1
Hi

We're a small packaged healthy snack company. We sell our products in a particular chain of stores and we get weekly sales data per store per product per week. Screenshot of data is attached.

We recently did a test where we applied coupons directly on to products in three stores (think of the little sticker coupons that say save $1 and you peel it off and scan it at checkout). The point of the test was to see if those little coupons resulted in a sales lift, and if so, what is the general conclusion on amount of lift.

Other things that come into play
  • There are about 20 weeks where our products were on promotion in all these stores. That would skew the data and I feel like I should filter those weeks out?
  • This test took place in the weeks in 12/8/18 through 12/29/18. Our products typically do not sell well those last two weeks of the year as people are not eating healthy at that time so it's not really fair to compare sales from the prior weeks, it should be probably be compared year over year?

To reiterate, the key question is: did those stores in question see a statistically significant lift in sales, and if so, how much? I would love to hear some thoughts on the best approach.

Thank you for reading!
 

Attachments

Last edited:
#9
Stores where test was conducted: 158, 20, 36

I attached a new file. Same data as before but I added a column for promo to mark the weeks ending where our products were on promo in ALL stores. Including those weeks is going to skew the data I would think so the new file makes it easy to filter those out.
 

Attachments

Miner

TS Contributor
#10
The promo is statistically significant, but you will have to make a judgment on whether it is of practical benefit as the effect was relatively small compared to the overall noise in the data. However, small changes do add up over time.

The first attachment is the statistical analysis using the year over year difference that shows the effect and the interaction (effect varies between stores). The second attachment is a graphical analysis showing the overall effect by store and the repeating time effect displayed on a control chart (used in industrial statistics).
 

Attachments

#11
Wow. Thank you for doing all that!! Just so I'm clear, did you filter out the weeks where our products were on promo in ALL stores and then compare the sales data from the coupon/test stores (i.e. store #'s 158, 20, 36) vs all the other stores for the weeks where there was no promo?
 
#13
Can I send you some free product in exchange for 30 minutes of your time to go over this? I would have sent you a private message about this, but I don't see that as an option in the forum.
 
#16
Thanks Miner. When I try to start a new conversation it just says I do not have permission to perform this action. My email address is verified but my account is pretty new so maybe that is the issue. I submitted a message to the administrator asking for help.