t-score sample sizes

#1
Hopefully someone can help me with this....

I am projecting forward two fish stock populations for a 10 year period with different levels of fishing mortality and am trying to compare mean population sizes at the end of each year to see if there is a statistical difference if fishing pressure is reduced.

The t-test results are confusing me though. I'm using a fisheries model to project the population forward and I can choose the number(n) of iterations/runs per year. So what I am left with to analyze are a number of arrays of n possible population sizes (as in the example below where n=4). I then conduct a t-test between the mean population size in 2008 for population A and population B and do the same for 2009. This leaves me with 2 t-scores, one for 2008 and one for 2009.

I have conducted a t-test with n=100, n=1000 and n=10,000. I would have expected the t-scores not to change much between each of these and was basically increasing n just to increase my confidence in the outcome. However, I found that at n=100 only about 3% of the time are the results significant. At n=1000 this rises to 35% and at n=10,000 everything is significant.

I have spent days trying to figure out why increasing n has this effect on the t-score but have not been able to get anywhere. If anybody can shed some light on this I'd be very grateful.

thanks in advance.

Barry

Population A Population B
2008 2009 2008 2009
27806 33694 35210 40738
41832 39478 21677 21851
16868 15653 35363 33170
45645 44100 25917 33957
 
#2
Generally speaking, the larger your sample size,the smaller the variance and standard deviation. the t score is "mean devided by standard deviation". Thus, the t score increases as SD decreases.

Phil
 
#3
So with the t-tests, once you increase the sample size will your t-score always become significant? In this case the level of significance increases in proportion with the number of samples? Surely it is not a reliable indicator of significance if all you have to do is increase the number of samples!!
 
#4
Are you talking sample size or number of samples?
As you increase sample size significance becomes greater.
There is a difference between effect size and significance. You can have a strong effect but large P value if your sample is small. Conversely, you can have a small effect and small P value if your sample is large. If you have a super large sample you will no doubt get a very tiny P value. But you should also look at the effect size. If the effect size is super tiny then it probably isn't particularly interesting, even though it is significant.


If an there really is an effect As you increase your sample size