Recently I had this idea, to study noise in fx markets. I wrote my own statistics library, and when I started to apply tests I noticed that the tests were not accurate at all, Bartlett and Levene giving quite opposite results. Thing is, I had no idea how to apply the tests - I don't know in how many groups should I break the time series - because the tests weren't devised to analyse time series. So I made an 100 value sample (the first column of the gear.dat found here: http://www.itl.nist.gov/div898/handb...n3/eda3581.htm). But I don't get the same results and I don't know wether to trust my own calculations or not. Even for the Levene test, the inverse F function that I have doesn't yield the same results. (My inverse F (5, 10, 0.05) yields 3.3258, but for (9,90,0.05) yields 0.9999 instead of 1.985595 and the code is translated from the ALGLIB.net package for c++). So, multiple head scratches. But first I'd like to see if the algos are correct. I am pretty sure, I've been a lot thru the code, yet I don't have the final confirmation.
Hmm.... I tried to find the book but it doesn't seem to be anywhere. The 1962 edition is available online, but these tests appeared in the 70s. I don't know... guess I could make with someone a sample check, but that ain't gonna tell where the errors are...
P.S. Can the bartlett score be negative? I mean I look at the formula, I know it can be, but it doesn't feel normal.
Well, it would seem unlikely (or doesn't feel normal as you put it) because Bartlett's statistic is Chi-square.
That said, situations can arise that don't feel normal such as:
Can someone obtain a negative R^2 associated with a regression model?
The answer is Yes. This can occur when we have a regression model without an intercept term...i.e. when we are regressing through the origin.
I debugged and found an error in the bartlett code - I was using stddev instead of variance - but after I corrected, I still get a slight different answer from the handbook. As for Levene, that's way, way farther than it should. So now I started to do them in Excel, and confront with the results from the code. However Bartlett code is simple - spans about one page of code - I don't think there are more errors in it... I could paste the code here, (because basic statistics are done in a class) but I don't think that would be of any help.
I started to make an example in excel, and what strikes me is that both VAR and STDEV in excel divide to (N-1) instead of N (!!)
... Most things divide by n-1 for estimates of the variance when you're using a sample. If you're working with population data then you could readjust it by multiplying by (n-1)/n and then you're on the scale you're use to but really dividing by n-1 is what is normally done. It makes the estimate unbiased.
Finally Bartlett works! It was a compiler glitch, does not like integer values in double formulas...
Can't say anything about the groups variance... Whatever method I apply, score seems to remain the same.
And now, heading on to Levene...
Last edited by quantster; 09-03-2010 at 11:41 AM.
This is the result for the sample of the EDA handbook:
2010.09.03 17:45:55 bartletttest (EURUSD,D1) Bartlett score : 20.78587342806486 Critical = 16.91897760462045
2010.09.03 17:45:55 bartletttest (EURUSD,D1) final denominator = 1+(1/(3*9)))*1.111111111111111-1/90 = 1.037037037037037*1.1=1.040740740740741
2010.09.03 17:45:55 bartletttest (EURUSD,D1) final numerator = 90 * -10.25257232720819--944.3642147572043 = 21.63270530846751
2010.09.03 17:45:55 bartletttest (EURUSD,D1) sum_denominator=1+ (1/9) = 1.111111111111111
2010.09.03 17:45:55 bartletttest (EURUSD,D1) sum_numerator=-850.1421220420303+9*-10.46912141279711 = -944.3642147572043
2010.09.03 17:45:55 bartletttest (EURUSD,D1) 8 :: sum_denominator=0.8888888888888891+ (1/9) = 1
2010.09.03 17:45:55 bartletttest (EURUSD,D1) 8 :: sum_numerator=-751.3659215052605+9*-10.97513339297442 = -850.1421220420303
2010.09.03 17:45:55 bartletttest (EURUSD,D1) 7 :: sum_denominator=0.7777777777777779+ (1/9) = 0.8888888888888891
2010.09.03 17:45:55 bartletttest (EURUSD,D1) 7 :: sum_numerator=-650.2179237896053+9*-11.23866641285059 = -751.3659215052605
2010.09.03 17:45:55 bartletttest (EURUSD,D1) 6 :: sum_denominator=0.6666666666666667+ (1/9) = 0.7777777777777779
2010.09.03 17:45:55 bartletttest (EURUSD,D1) 6 :: sum_numerator=-563.0305991384842+9*-9.687480516791236 = -650.2179237896053
2010.09.03 17:45:55 bartletttest (EURUSD,D1) 5 :: sum_denominator=0.5555555555555556+ (1/9) = 0.6666666666666667
2010.09.03 17:45:55 bartletttest (EURUSD,D1) 5 :: sum_numerator=-479.9311882489051+9*-9.233267876619895 = -563.0305991384842
2010.09.03 17:45:55 bartletttest (EURUSD,D1) 4 :: sum_denominator=0.4444444444444444+ (1/9) = 0.5555555555555556
2010.09.03 17:45:55 bartletttest (EURUSD,D1) 4 :: sum_numerator=-392.0472169210186+9*-9.764885703098498 = -479.9311882489051
2010.09.03 17:45:55 bartletttest (EURUSD,D1) 3 :: sum_denominator=0.3333333333333333+ (1/9) = 0.4444444444444444
2010.09.03 17:45:55 bartletttest (EURUSD,D1) 3 :: sum_numerator=-291.986253053237+9*-11.11788487419796 = -392.0472169210186
2010.09.03 17:45:55 bartletttest (EURUSD,D1) 2 :: sum_denominator=0.2222222222222222+ (1/9) = 0.3333333333333333
2010.09.03 17:45:55 bartletttest (EURUSD,D1) 2 :: sum_numerator=-192.4993968263334+9*-11.05409513632262 = -291.986253053237
2010.09.03 17:45:55 bartletttest (EURUSD,D1) 1 :: sum_denominator=0.1111111111111111+ (1/9) = 0.2222222222222222
2010.09.03 17:45:55 bartletttest (EURUSD,D1) 1 :: sum_numerator=-97.89243028425207+9*-10.51188517134237 = -192.4993968263334
2010.09.03 17:45:55 bartletttest (EURUSD,D1) 0 :: sum_denominator=0+ (1/9) = 0.1111111111111111
2010.09.03 17:45:55 bartletttest (EURUSD,D1) 0 :: sum_numerator=0+9*-10.87693669825023 = -97.89243028425207
2010.09.03 17:45:55 bartletttest (EURUSD,D1) sp=3.526666666666663e-005
2010.09.03 17:45:55 bartletttest (EURUSD,D1) si[9]=2.840000000000005e-005 ; sp=3.242666666666663e-005+9*2.840000000000005e-005/(100-10) = 3.526666666666663e-005
2010.09.03 17:45:55 bartletttest (EURUSD,D1) si[8]=1.712222222222225e-005 ; sp=3.07144444444444e-005+9*1.712222222222225e-005/(100-10) = 3.242666666666663e-005
2010.09.03 17:45:55 bartletttest (EURUSD,D1) si[7]=1.315555555555558e-005 ; sp=2.939888888888884e-005+9*1.315555555555558e-005/(100-10) = 3.07144444444444e-005
2010.09.03 17:45:55 bartletttest (EURUSD,D1) si[6]=6.205555555555568e-005 ; sp=2.319333333333327e-005+9*6.205555555555568e-005/(100-10) = 2.939888888888884e-005
2010.09.03 17:45:55 bartletttest (EURUSD,D1) si[5]=9.773333333333265e-005 ; sp=1.342000000000001e-005+9*9.773333333333265e-005/(100-10) = 2.319333333333327e-005
2010.09.03 17:45:55 bartletttest (EURUSD,D1) si[4]=5.743333333333343e-005 ; sp=7.676666666666664e-006+9*5.743333333333343e-005/(100-10) = 1.342000000000001e-005
2010.09.03 17:45:55 bartletttest (EURUSD,D1) si[3]=1.48444444444443e-005 ; sp=6.192222222222235e-006+9*1.48444444444443e-005/(100-10) = 7.676666666666664e-006
2010.09.03 17:45:55 bartletttest (EURUSD,D1) si[2]=1.582222222222225e-005 ; sp=4.610000000000009e-006+9*1.582222222222225e-005/(100-10) = 6.192222222222235e-006
2010.09.03 17:45:55 bartletttest (EURUSD,D1) si[1]=2.721111111111116e-005 ; sp=1.888888888888892e-006+9*2.721111111111116e-005/(100-10) = 4.610000000000009e-006
2010.09.03 17:45:55 bartletttest (EURUSD,D1) si[0]=1.888888888888892e-005 ; sp=0+9*1.888888888888892e-005/(100-10) = 1.888888888888892e-006
EDA Handbook test:
BARTLETT TEST
(STANDARD DEFINITION)
NULL HYPOTHESIS UNDER TEST--ALL SIGMA(I) ARE EQUAL
TEST:
DEGREES OF FREEDOM = 9.000000
TEST STATISTIC VALUE = 20.78580
CUTOFF: 95% PERCENT POINT = 16.91898
CUTOFF: 99% PERCENT POINT = 21.66600
CHI-SQUARE CDF VALUE = 0.986364
NULL NULL HYPOTHESIS NULL HYPOTHESIS
HYPOTHESIS ACCEPTANCE INTERVAL CONCLUSION
ALL SIGMA EQUAL (0.000,0.950) REJECT
Wonder why the 5th decimal is not correct...
However, send over the sample. I'll find a way to load and test.
Last edited by quantster; 09-03-2010 at 11:50 AM. Reason: forgot something
Okay, I've got to run and teach class in just a few minutes. I'll do it after I finish and note that I can't remember if they actually have the sample data or just the summary statistics i.e. MSerror, samples sizes, number of groups, bla, bla, bla.
Follow-up comment: I checked the book and they only provide the summary statistics and not the actual data and thus I don't think that it is going to do you any good.
Levene still doesn't work. It seems okay to agree with Excel stuff, but for the handbook test it yields other results:
PL 0 levenetest (EURUSD,D1) 20:22:54 Added to sum_denominator: 1.520999999999964e-005 = 1.520999999999964e-005
DJ 0 levenetest (EURUSD,D1) 20:22:54 Added to sum_denominator: 9.999999999999021e-007 = 1.620999999999954e-005
IH 0 levenetest (EURUSD,D1) 20:22:54 Added to sum_denominator: 8.999999999999717e-006 = 2.520999999999926e-005
NF 0 levenetest (EURUSD,D1) 20:22:54 Added to sum_denominator: 4.410000000000218e-006 = 2.961999999999947e-005
ED 0 levenetest (EURUSD,D1) 20:22:54 Added to sum_denominator: 9.000000000000315e-006 = 3.861999999999979e-005
PR 0 levenetest (EURUSD,D1) 20:22:54 Added to sum_denominator: 4.000000000000207e-006 = 4.262e-005
...
PE 0 levenetest (EURUSD,D1) 20:22:54 Added to sum_denominator: 9.891025000000302e-006 = 0.00145635790000001
LG 0 levenetest (EURUSD,D1) 20:22:54 Added to sum_denominator: 3.822025000000221e-006 = 0.00146017992500001
CI 0 levenetest (EURUSD,D1) 20:22:54 Added to sum_denominator: 4.601025000000202e-006 = 0.00146478095000001
KJ 0 levenetest (EURUSD,D1) 20:22:54 Added to sum_denominator: 1.486102499999968e-005 = 0.00147964197500001
ML 0 levenetest (EURUSD,D1) 20:22:54 Added to sum_denominator: 9.891025000000302e-006 = 0.00148953300000001
PP 0 levenetest (EURUSD,D1) 20:22:54 group 0 : Y[0]=0.9985500000000001 ziavg[0]=0.003550000000000003
KS 0 levenetest (EURUSD,D1) 20:22:54 group 1 : Y[1]=0.9967999999999998 ziavg[1]=0.003079999999999977
CF 0 levenetest (EURUSD,D1) 20:22:54 group 2 : Y[2]=0.9953500000000002 ziavg[2]=0.006979999999999953
OJ 0 levenetest (EURUSD,D1) 20:22:54 group 3 : Y[3]=1.00095 ziavg[3]=0.004244999999999999
NM 0 levenetest (EURUSD,D1) 20:22:54 group 4 : Y[4]=0.9965500000000001 ziavg[4]=0.003594999999999998
PR 0 levenetest (EURUSD,D1) 20:22:54 zij[0]=0.007449999999999957
DJ 0 levenetest (EURUSD,D1) 20:22:54 zij[1]=0.002550000000000052
NN 0 levenetest (EURUSD,D1) 20:22:54 zij[2]=0.0005500000000000505
CE 0 levenetest (EURUSD,D1) 20:22:54 zij[3]=0.001449999999999951
EM 0 levenetest (EURUSD,D1) 20:22:54 zij[4]=0.006550000000000056
JE 0 levenetest (EURUSD,D1) 20:22:54 zij[5]=0.005550000000000055
....
DD 0 levenetest (EURUSD,D1) 20:22:54 zij[95]=0.0004499999999999504
JL 0 levenetest (EURUSD,D1) 20:22:54 zij[96]=0.005550000000000055
DD 0 levenetest (EURUSD,D1) 20:22:54 zij[97]=0.001449999999999951
GM 0 levenetest (EURUSD,D1) 20:22:54 zij[98]=0.007449999999999957
JF 0 levenetest (EURUSD,D1) 20:22:54 zij[99]=0.0004499999999999504
QL 0 levenetest (EURUSD,D1) 20:22:54 zdbldot=0.004289999999999986
MS 0 levenetest (EURUSD,D1) 20:22:54 Sum numerator calculation
GE 0 levenetest (EURUSD,D1) 20:22:54 Added to sum numerator : 20*(0.003550000000000003-0.004289999999999986)^2 : 1.095199999999948e-005 = 1.095199999999948e-005
PI 0 levenetest (EURUSD,D1) 20:22:54 Added to sum numerator : 20*(0.003079999999999977-0.004289999999999986)^2 : 2.928200000000041e-005 = 4.023399999999989e-005
NM 0 levenetest (EURUSD,D1) 20:22:54 Added to sum numerator : 20*(0.006979999999999953-0.004289999999999986)^2 : 0.0001447219999999965 = 0.0001849559999999964
EM 0 levenetest (EURUSD,D1) 20:22:54 Added to sum numerator : 20*(0.004244999999999999-0.004289999999999986)^2 : 4.04999999999764e-008 = 0.0001849964999999964
QI 0 levenetest (EURUSD,D1) 20:22:54 Added to sum numerator : 20*(0.003594999999999998-0.004289999999999986)^2 : 9.660499999999646e-006 = 0.000194656999999996
QG 0 levenetest (EURUSD,D1) 20:22:54 sum_numerator = 0.000194656999999996 * 95 = 0.01849241499999962
CJ 0 levenetest (EURUSD,D1) 20:22:54 sum_denominator = 0.00148953300000001 * 4 = 0.00595813200000004
ON 0 levenetest (EURUSD,D1) 20:22:54 Levene score : 3.103726973487579 Critical = 1.000000000000116
EDA handbook test:
LEVENE F-TEST FOR SHIFT IN VARIATION
(CASE: TEST BASED ON MEDIANS)
1. STATISTICS
NUMBER OF OBSERVATIONS = 100
NUMBER OF GROUPS = 10
LEVENE F TEST STATISTIC = 1.705910
So...pretty far
Last edited by quantster; 09-05-2010 at 02:30 PM.
I was mistaken ; I set my Levene to work with averages; when I set it to work with medians, I get a score of 2.7615, so not yet correct...
2010.09.07 11:58:14 levenetest (USDJPY,M5) Levene score : 1.705917693000894 Critical = 0.9999998983301629
2010.09.07 11:58:14 levenetest (USDJPY,M5) sum_denominator = 0.001331599999999996 * 9 = 0.01198439999999996
2010.09.07 11:58:14 levenetest (USDJPY,M5) sum_numerator = 0.0002271599999999982 * 90 = 0.02044439999999984
And now, the inverse F...
Tweet |