Reliability question

#1
I am doing some research at the moment using something called transcranial magnetic stimulation, which involves delivering stimulation to the area of the brain controlling the muscle, and measuring the response in the muscle.

The response fluctuates between every stimulation, so we have to take several in order to get an accurate measure representing the true value of the response. The test I'm doing at the moment is working at the minimum number of stimulations to get an reliable estimate of the response in the muscle.

However, I am not sure how I go about analyzing that data or what tests I should implement. My supervisor has suggested bootstrapping however I am unsure whether this is appropriate or how to go about doing it. Any help would be much appreciated.
 

Miner

TS Contributor
#2
This sounds like a measurement system analysis. This varies depending on your field of study.

I work in industrial statistics where we break measurement variation into the following categories: 1) Bias (accuracy), 2) Linearity (bias across the measurement range), 3) Stability (bias over time), 4) Repeatability (variation between multiple measurements taken by one person) & Reproducibility (variation between different people taking the same measurement). Items 1- 3 affect the mean of the measurement while item 4 affects the variability of the measurement.

Your question is posed as a variability issue, so a test for repeatability and reproducibility is advised in order to quantify this variability. If that variability is unacceptable, you can reduce it by changing the test, the test procedure, having only one person take the measurements, etc. You can also reduce the standard deviation by averaging multiple measurements. The standard deviation of the mean measurement is equal to the standard deviation of the individual measurements divided by the square root of the sample size. That is, the standard deviation of the mean of 4 repeat measurements is one half that of the individual measurements.

You know your field better than I, but I would watch out for potential crossover effects, time dependency and autocorrelation with your data.