I am new to the forum and learning the stats I need as I go I have done some parametric stats in undergrad but never dealt with non-parametric stats before so I am hoping to get some direction and pick up some language to help me describe the problems better. Any assistance in both directions would be greatly appreciated.

We use a lot of online surveys for data collection. One question type in particular asks for a integer rating on a scale from 1 to 6 covering the range from strongly dis-agree to strongly agree).

We have a report tool that calculates the frequency histograms for the ratings for a user defined population (rarely close to normal) and the average rating. We do the same analysis for a variety of groups each 3 months and look for material changes in the statistics period on period.

I want to be able put some rigour to detecting a material change in average between periods for the same group. The analogy with normal distribution sampling and the distribution of sample averages came to mind where, I understand, I could look at the probability that the current quarters average could have been randomly selected from the distribution of average ratings from last period. In a normally distributed, continuous world this is neat as the distribution of averages is easy to calculate. However, with our non-continuous distribution of discrete rating I am at a loss as to how to approach the problem of detecting material changes.

I would be very grateful to anyone who could provide some insight to me in how to approach this problem or other ideas/tests etc that may achieve the same desired outcome of finding appropriate statistics to help identify material changes in the average or perhaps the pmf.

Many thanks

Matt