How to combine several data sets

#1
I have hourly logs of web server response times that I need to combine into a daily summary. I don't have the original source data or I would just compute the daily summary using the source. Each hourly data set has the following fields already computed: sample size, mean, std dev, median, and 90th & 95th percentile.

I want the summary to contain the same fields as the hourly data set. Any pointers?

I did some hunting around for combining data sets but only found explanations for weighted average. Any pointers or links to explanations are appreciated.

As an extra bonus, the data sets are in Excel so even better if you're able to frame the response in Excel.

Thanks!
 

JohnM

TS Contributor
#2
You can either write a macro to run through the hourly data and produce a set that represents the weighted averages for whole days, or you can just use Excel spreadsheet formulas to do it "manually."
 
#3
Thanks for the followup. What I was saying is that of all those fields I only know how to compute the weighted average. What I don't is how to combine the data sets to get the combined std dev, median, 90th & 95th percentile. Any ideas?

You can ignore the Excel context. I first need to understand how to compute it mathematically, then I can next tackle it in Excel. Thanks
 

JohnM

TS Contributor
#4
For the std devs, square them to get the variances and compute a weighted avg variance, then take the square root to get the std dev.

For the percentiles (including the median), I don't know of a good, valid method.....so I would just report the range of values (lowest median, highest median, lowest 90th pct, highest 90th pct, etc.)