Adjusting confidence intervals for multiple comparisons

[hlsmith]

Everything put together:

data test;
input mean sd n;
cards;
10 2.5 40
15 2.0 37
08 2.3 45
17 3.4 35
11 2.9 41
;
run;

proc sql; create table test as select *, monotonic() as class from test; quit;
proc sql noprint; select count(distinct(class)) into :n from test; quit;
proc sql noprint; select fact(count(distinct(class)))/((fact(2))*(fact(count(distinct(class)) - 2))) into :nC2 from test; quit;

proc plan;
	factors test_number=&nC2. ordered
	class= 2 of &n. comb ;
	ods output plan=combinations;
	run; quit;

proc sql;
create table temp1 as
select
	c.test_number,
	c.class1,
	t.mean as mean1,
	t.sd as sd1,
	t.n as n1
from
	combinations c
	left join
	test t
	on c.class1 = t.class
; quit; 

proc sql;
create table temp2 as
select
	c.test_number,
	c.class2,
	t.mean as mean2,
	t.sd as sd2,
	t.n as n2
from
	combinations c
	left join
	test t
	on c.class2 = t.class
; quit; 

proc sql;
create table joined as
select
	t1.*,
	t2.*
from
	temp1 t1,
	temp2 t2
where
	t1.test_number = t2.test_number
order by
	t1.test_number
; quit;

data combineTest2;
  set joined;
  f=finv(0.95,&p,n1+n2-&p-1); 
  t=tinv(1-0.025/&p,n1+n2-2); 
  sp=((n1-1)*sd1+(n2-1)*sd2)/(n1+n2-2); 
  losim=mean1-mean2-sqrt(&p*(n1+n2-2)*f*(1/n1+1/n2)*sp/(n1+n2-&p-1)); 
  upsim=mean1-mean2+sqrt(&p*(n1+n2-2)*f*(1/n1+1/n2)*sp/(n1+n2-&p-1)); 
  lobon=mean1-mean2-t*sqrt((1/n1+1/n2)*sp); 
  upbon=mean1-mean2+t*sqrt((1/n1+1/n2)*sp); 
  run;
proc print;
  run;

[Dason]

I wrapped your code in [code] [/code] tags for you so it displays better.

[lancearmstrong1313]

Can anyone explain to me why for the Bonferonni CI, the code is doing 1-.025/k instead of 1-.05/k?

[hlsmith]

To me it seems to be because otherwise it would be 90% CI.

95%-5% = 90%

See this link for another example:
http://www.statisticslectures.com/to...ndentsamplest/

However, my basic question is why when I conduct the t-test:

data test;
input Category  _TYPE_  _FREQ_  _STAT_$ YourVar;
datalines;
1           0       6      N         40
1           0       6      MEAN    10
1           0       6      STD      2.5
2           0       6      N         37
2           0       6      MEAN    15
2           0       6      STD      2.0
;
run;
proc ttest data=test;
class category;
var YourVar;
run;

are the CIs: -6.0332, -3.9668 and the Bonferroni are -5.9293, -4.07073. I would have imagined that they should have widened on both ends, so the -5.9293 would actually be smaller than -6.0332. I am probably missing something simple (such as how they both are being caluculated).

[Dason]

To make a % confidence interval you typically use the quantile of the distribution of interest. This is why it's 1-alpha/2 instead of 1-alpha.

[lancearmstrong1313]

Oh yeah duh, because it's technicailly (.05)/(2*k) so the code is just canceling out the .05/2 part. I'm used to seeing it with the .05 in the numerator and 2 in the denominator that I didn't notice the code just canceled the .05/2 part - I'm dumb