p-charts for complaint monitoring

[statgirl26]

I am in a position where I am looking at complaints for different products. It has been tradition to monitor the complaints using a modified p-chart approach with a rolling baseline. The monitoring is done on monthly basis over a 15 month period with 9 months used as a baseline period and 6 months used as a signaling period. The complaints are examined as a rate: complaints/number of units sold in a month. I actually have a few questions about this method:
1. The standard textbook approach to calculating the upper control limit is done by multiplying the standard deviation by 3. What amount of error are we accepting by only using a 9 month baseline period?
2. We have had outside consultants examine our approach, and they suggest using a binomial distribution for calculating an upper control limit. Is this approach appropriate given that our data is continuous and a binomial distribution is discrete?
3. Unfortunately our method is not without limitations. Our number of units sold per month is not a good denominator since it does not tell us how many products are actually in use. It also creates false alarms when sales are volatile. Does anyone have any suggestions of a better way to monitor complaints or to adjust the denominator?
Any insight would greatly be appreciated.

[JohnM]

1. The standard textbook approach to calculating the upper control limit is done by multiplying the standard deviation by 3. What amount of error are we accepting by only using a 9 month baseline period?

This is highly dependent on how your "rate of complaints process" runs over the long term. How long do you feel it should be? Do you have any data to support that it should be longer? Maybe it should be as long as your "sales cycle" or something that encompasses any seasonality in the sales trends?

2. We have had outside consultants examine our approach, and they suggest using a binomial distribution for calculating an upper control limit. Is this approach appropriate given that our data is continuous and a binomial distribution is discrete?

Actually, your data is discrete, because it is countable (you can count complaints, and you can count # of units sold). If you are talking about large numbers, you may be able to use the normal approx to the binomial.

3. Unfortunately our method is not without limitations. Our number of units sold per month is not a good denominator since it does not tell us how many products are actually in use. It also creates false alarms when sales are volatile. Does anyone have any suggestions of a better way to monitor complaints or to adjust the denominator?

Here you'll need to do some market research - but I would assume that units sold would be about as close as you can get to # of products in use...

[statgirl26]

Thank you for your response.
To answer your question, there is no seasonality or sales trend in our data. The best answer I can give you is that when a new product is put on the market, sales drastically increase. Also, when there is a new marketing tactic sales also tend to increase.
I felt that the baseline period should be longer because we are getting many false signals due to the volatility in our sales data. Also, I don't feel like our baseline is ever during a period of steady state.
Again to address the issue of our denominator: we are only using number of units sold in the current month. Unfortunately our products sold in a month do not reflect the number of products actually used in a month because of a delay in shipping, etc.
Is there a better approach to monitoring the rate of complaints over time that would avoid some of these issues?

[JohnM]

I Googled the terms SPC and "customer complaints" and got some interesting links.

These resources may contain info from people / companies that have confronted these issues before - one example I saw did not attempt to track the rate of complaints, rather they just tracked the number of complaints, and categorized them by type. Once they got enough data and saw which categories were the most common, they fixed them, and that led to fewer complaints, and so on....

You may just need to do some basic research into how others have tackled the problem. Maybe you just need to change the approach - remember, your goal is not the statistical method, it's to get rid of the complaints.

Google Links

http://www.spcforexcel.com/customercomplaintspc.htm

[statgirl26]

Thanks again for your response.
Back to my first questions:
I understand that the counts are discrete, but we then are using the counts to create a proportion that is graphed as a rate. Isn't the rate continuous? If so, then why should the threshold be calculated using a discrete distribution?
Is there a formula I can use to calculate type 2 error for each of my models? I've been doing a literature review, and the best thing I can find is to use an operating characteristic curve. Do you know anything about this? Isn't the curve based off of the normal distribution?

[JohnM]

It's not so much a question of discrete vs continuous, it's more about using the correct underlying distribution.

The normal distribution is a good approximation of the binomial only if n*p and n*q are both at least 5. If that's not the case, then the binomial would be preferred.

OC curves are based on whatever is the appropriate underlying distribution. I've seen them based on the normal, and I've seen them based on the binomial.

[statgirl26]

I've been looking into the formulas for calculating type II error for p-charts and I don't think it is possible to use OC curves. This is because the formula requires both a sample size and a population size. Since we are using monthly sales as an estimate for the population, I don't see how this is applicable since we are not sampling. Do you have any suggestions of any other way I could calculate type II error?