Seeking basic analysis help

Seeking analysis help (defect rates)

I have a question regarding an analysis of the number of defects found in products over a certain time period and whether or not a method to fix these defects is working.

The data are split into two time periods: before and after the “fix” was implemented.
The Before period consists of roughly a couple years of data. The After period is significantly less – only about 3-4 months. To make a proper assessment of the level of defects in both periods, I’m sure a proportional sample of the Before period will need to be taken. Unfortunately, I am not able to wait longer for more data to appear in the After period for a stronger analysis.

I am able to retrieve the stats for any data I choose (population mean, sample mean, etc. for any relevant data), so I don’t see how to apply statistical methods to inquire about information I already have. Furthermore, I am almost certain the data are not normally distributed so I guess a transform may be needed.

I guess, in short, my question is: how can I tell if things are actually getting better? All I can think of is to choose a random sample from both periods and compare the weighted averages of defects of both samples (just by quick glance I am inclined to say, “yes” since the average number of defects is significantly lower. Though I understand a great number of factors could be coming into play for this lower defect rate other than simply applying a "fix.") My problem is others tell me they are looking for some degree of certainty or confidence interval and to use a null/alternative hypothesis approach, but I have no idea how to use these in this type of setting. Statistics is not my trade (I studied pure mathematics in school.) I can definitely learn whatever is required but I don’t know what path to take.

I greatly appreciate any help you can provide. Please let me know if you need any additional information. Thanks!
Last edited:


Ninja say what!?!
I'd look into survival analysis or rates analyses. In essence, what you're asking is whether the rate of defects is the same now as it was before.

Sorry I can't be more detailed. However, this subject isn't something that I can explain simply in one post. Hope that helps though.