Sample size calculation for a survey on time management

#1
Hello everyone,

We conducted a time management survey (Time management skills test) on medical teachers in our and another neighboring medical college. We were able to collect 123 responses for the time management survey. The survey has 100 points for scoring. Scores below 41 are considered excellent. Scores between 41 and 56 are considered good while scores between 56 and 100 are considered poor. The mean of all the medical teachers was 55.44.

Now we have submitted the paper to a journal and the editorial board have commented that the survey does not have adequate sample size. The sample size calculation according to most resources for an unknown population is 370. However i see papers published all the time in medical journals which have between 100 and 200 respondents. Am i missing something ? Kindly guide on how to do the sample size calculation in this particular case and what , if anything, we can write back to the editor to increase our chances of the paper getting accepted.

Many thanks.
 
Last edited:
#2
While searching about this issue online i have found a document that recommends using this formula to find out sample size

N= 4*SD2/M2

That is N is equal to 4 into SD squared divided by M squared where M is the margin of error. So using this equation, the sample size turns out to be 36 (As the SD is 15 and the margin of error is 5). Can this equation be used to assess sample size in this case ?
 
Last edited:

Karabiner

TS Contributor
#3
Unfortunately I do not know this equation and what exactely is meant by margin of
error here. Usually, one can use a power analyis software, such as G*power for example.,
or a power analysis online calculator, such as https://clincalc.com/stats/samplesize.aspx.
But n=36 looks almost certainly inadequate.

You have to determine which effect size (e.g. standardized mean difference, as
in "Cohen's d") you expect between populations. Based on this, you can calculate
which sample size you need in order to have an 80% chance to see a statistically
significant test result (the 80% is an often used threshold for statistical power,
but you can use a higher or lower threshold).

For example, for a "medium" effect in the population (Cohen's d = 0.5), you need
about 130 participants in order to achieve an 80% statistical power. For a smaller
effect, such as d=0.3, you'd need about 350 participants

If the editors assume that the effects in the population are less than medium sized,
then your study was underpowered. If, on the other hand, you can argue that
the expected effect was indeed medium sized, then your sample size could be
adequate.

With kind regards

Karabiner
 
#4
Unfortunately I do not know this equation and what exactely is meant by margin of
error here. Usually, one can use a power analyis software, such as G*power for example.,
or a power analysis online calculator, such as https://clincalc.com/stats/samplesize.aspx.
But n=36 looks almost certainly inadequate.

You have to determine which effect size (e.g. standardized mean difference, as
in "Cohen's d") you expect between populations. Based on this, you can calculate
which sample size you need in order to have an 80% chance to see a statistically
significant test result (the 80% is an often used threshold for statistical power,
but you can use a higher or lower threshold).

For example, for a "medium" effect in the population (Cohen's d = 0.5), you need
about 130 participants in order to achieve an 80% statistical power. For a smaller
effect, such as d=0.3, you'd need about 350 participants

If the editors assume that the effects in the population are less than medium sized,
then your study was underpowered. If, on the other hand, you can argue that
the expected effect was indeed medium sized, then your sample size could be
adequate.

With kind regards

Karabiner
OK thank you very much for a detailed reply. This is very helpful. Is there anyway i could reference the fact that for a medium effect in the population, a sample of 130 is needed. If i could quote a reference to the editors, it would make life easy. Please guide what reference i can provide to support this assertion. Many thanks again.
 
#7
There is no reference I know of. I calculated this using https://clincalc.com/stats/samplesize.aspx .

With kind regards

Karabiner
Can you very kindly guide on how you reached at the sample size of 130 for a moderate effect. The calculator you referenced asks for an anticipated mean from the previous and planned study but i cannot figure how to calculate sample size by the effect size (moderate in our case). Your advice will be much appreciated.
 

Karabiner

TS Contributor
#8
A moderate effect size would often be assumed with -> Cohen's d = 0.5, i.e. the mean difference is half
the size of the standard deviation of the total group.
For a rough estimation, I put in artificial numbers and used the SD of the first group only, i.e. mean 100
and SD 10, versus mean 105.

With kind regards

Karabiner
 
#9
A moderate effect size would often be assumed with -> Cohen's d = 0.5, i.e. the mean difference is half
the size of the standard deviation of the total group.
For a rough estimation, I put in artificial numbers and used the SD of the first group only, i.e. mean 100
and SD 10, versus mean 105.

With kind regards

Karabiner
OK that makes it more clear. Many thanks again for your much appreciated guidance.
 

Karabiner

TS Contributor
#10
If you need a citable source for the reviewers, then you can
carry out the calculations with the g*power freeware by Faul &
Erdfelder.

With kind regards

Karabiner
 
#11
If you need a citable source for the reviewers, then you can
carry out the calculations with the g*power freeware by Faul &
Erdfelder.

With kind regards

Karabiner
Thank you for the tip. The issue with G*power is that it can be difficult to select which options to choose from a list of given options. For example, in this study (analyzing time management behaviors in medical teachers ) it is difficult to see which of the options apply from the given set of options, as it the study is about finding out a mean of a certain sample. Can you very kindly guide on how to do g*power calculations on this and similar studies where a simple mean or prevalence is needed to be established.
 
#12
If you need a citable source for the reviewers, then you can
carry out the calculations with the g*power freeware by Faul &
Erdfelder.

With kind regards

Karabiner
I am actually planning another study to assess the prevalence of anxiety and depression in patients with chronic liver disease. Can you kindly guide on how i can run g*power calculation for that study, as i am not sure which of the given options actually apply. Thanks.
 
#14
Under the normality assumption.
If you want to compare to the population mean you need to choose one sample t-test.
If you want to compare to another sample you need to run two-sample t-test. (like in the screenshot example)

If you know the standard deviation you can decide what unstandardized effect size you want to be able to discover.
For example, a change of 1gram if the population average is 100grm you want to be able to identify if the average is bigger than 101gr or smaller than 99gr

If you don't know the standard deviation that you need to use a standardized effect size (cohen's) as dear Karabiner explained.
( 0.2 small effect, 0.5 medium effect, 0.8 large effect. the smaller the required effect the bigger the sample size)

You can use the following to get power per sample size chart. http://www.statskingdom.com/sample_size_t_z.html two sample t power2.jpg
 
Last edited:
#15
Under the normality assumption.
If you want to compare to the population mean you need to choose one sample t-test.
If you want to compare to another sample you need to run two-sample t-test. (like in the screenshot example)

If you know the standard deviation you can decide what unstandardized effect size you want to be able to discover.
For example, a change of 1gram if the population average is 100grm you want to be able to identify if the average is bigger than 101gr or smaller than 99gr

If you don't know the standard deviation that you need to use a standardized effect size (cohen's) as dear Karabiner explained.
( 0.2 small effect, 0.5 medium effect, 0.8 large effect. the smaller the required effect the bigger the sample size)

You can use the following to get power per sample size chart. http://www.statskingdom.com/sample_size_t_z.html View attachment 1023
OK. Thank you very much for a detailed reply. Highly appreciated.