I am testing the effect of a new in-store promotion technique (treatment). It is applied during product promotions, as a compliment to regular promo displays, to further improve sales.

I have the treatment in only one store but on several products promotions in that store (I have observations on more than a 1000 different products). My goal is to estimate what additional uplift/drop in sales could be achieved due to the treatment. To do that I build regression model for a product’s sale in the test stores (using sales data of multiple similar control stores. The control stores have only the conventional promo activities during the promo period). Using this model I predict what the expected weekly sales for the promo period would be if there was only conventional promo activities and no treatment. Then I compare this expected with the actual sales and calculate the % uplift. An example is shown the table below.

Col1: Product

Col2: Expected sales in the promotional period (fitted value)

Col3: Actual sales with the treatment

Col4: % Uplift or Drop from the expected

item 1 X1 (ex. 100) A1 (ex: 150) 50

item 2 X2 A2 -9

item 3 X3 A3 9

item 4 X4 A4 29

item 5 X5 A5 76

item 6 X6 A6 -31

item 7 X7 A7 11

item 8 X8 A8 45

item 9 X9 A9 79

item 10 X10 A10 71

… … …

… ... …

item n Xn An ...

I have plotted the frequency distribution of the % uplift column and it looks like a bell curve. I have mean and median % uplift estimate. I want to have some kind of confidence around the point estimate for % uplift. I am not sure if I can do that and if so how? There is the regular t-test I am thinking about but then these uplift percentages are not just outcomes of a random experiment, they are outcomes based on my model. Would it still be okay to use t-distribution and build a certain confidence interval around my point-estimate?

Thanks for taking the time to review my question! I will greatly appreciate your suggestions!