etenebris
01-15-2008, 03:43 AM
First, please note I don't have a statistics background, so any help is greatly appreciated.
For Web based task completion rate analysis, I have been using Adjusted Wald error margins at X% confidence.
I need to know the confidence level when I combine these margins.
Example:
I have an A/B split test. A has 20 completions from 60 visitors. B has 22 completions from 50 visitors.
Using Adjusted Wald error margins at 68% confidence (z=1), I get 40.3% > A > 28.4% and 51.2% > B > 37.7%
Using these error margins, i see a relative improvement from A to B as being between -6.52% and 80.06%
(37.7%/40.3%) = 93.48%, -1 = -6.52%
(51.2%/28.4%) = 180.06%, -1 = 80.06%
I know that the chance of the conversion rate being at the maximum error range of one test and the minimum error range of the other test at the same time will be different, so the new error margin does not conform to the same 68% (z=1) confidence.
I am looking for the confidence that the relative improvement from A to B will lie within this range.
Have I made mistakes in my previous assumptions? Any tips on where I go from here?
For Web based task completion rate analysis, I have been using Adjusted Wald error margins at X% confidence.
I need to know the confidence level when I combine these margins.
Example:
I have an A/B split test. A has 20 completions from 60 visitors. B has 22 completions from 50 visitors.
Using Adjusted Wald error margins at 68% confidence (z=1), I get 40.3% > A > 28.4% and 51.2% > B > 37.7%
Using these error margins, i see a relative improvement from A to B as being between -6.52% and 80.06%
(37.7%/40.3%) = 93.48%, -1 = -6.52%
(51.2%/28.4%) = 180.06%, -1 = 80.06%
I know that the chance of the conversion rate being at the maximum error range of one test and the minimum error range of the other test at the same time will be different, so the new error margin does not conform to the same 68% (z=1) confidence.
I am looking for the confidence that the relative improvement from A to B will lie within this range.
Have I made mistakes in my previous assumptions? Any tips on where I go from here?