Distribution of p-values

#1
Hello

By way of some background, I have a data set with 160 observations. Each observation is a patient and the variable of interest is the number of appointments each had over a 5 month period. The question is whether there has been a decrease in patient appointments given what we could expect from their previous history. I'm comparing the number of appointments to the outcome of a model that predicts, based on each patients previous history, how many appointments we can expect in five months.

As the expected numbers are the output of a random model I ran the analysis 10,000 times and with each iteration I carry out a wilcoxon test in R with an alpha = 0.1, the alternative being less than the expected and I note the p value.

I now have 10,000 p values and find that 6790 are below alpha (0.1). Is it a simple case of running a binomial test (x= 6790, n = 10000, prob of success each trail = 0.1) to determine if these results are themselves significant and conclude that it would be unlikely to have so many significant results if there was no difference?

Andrew
 
#3
Thanks Englund.

If the p values are uniformly distributed under the null, should the probability of success in the binomial test be 0.5 rather than the alpha used (0.1)?

Thanks again for the help.