Accuracy of a measurement device

#1
Good afternoon,

I am working with a device that measures the internal dimensions of a piece. The device takes 27 different measurements, each one corresponding to a different part of the piece. The values obtained range from about 130.30mm to 129.30mm, normally decreasing at each measurement. I was trying to estimate how accurate this device can be by measuring the same piece many times and comparing the values. But I am not sure which test would show me this information. If anyone could help me I would be thankful. Follow one example of three different measurements made (I am actually planing to have about 10 measurements to compare):

Measurement 1 Measurement 2 Measurement 3
1 130.24 130.22 130.30
2 130.33 130.41 130.33
3 130.33 130.38 130.36
4 130.35 130.34 130.36
5 130.33 130.30 130.34
6 130.26 130.24 130.28
7 130.19 130.20 130.21
8 130.11 130.12 130.12
9 130.08 130.05 130.06
10 130.00 130.00 130.00
11 129.92 129.92 129.92
12 129.86 129.87 129.87
13 129.82 129.83 129.85
14 129.85 129.81 129.85
15 129.78 129.75 129.78
16 129.72 129.71 129.73
17 129.67 129.67 129.68
18 129.64 129.66 129.65
19 129.59 129.63 129.59
20 129.57 129.60 129.56
21 129.59 129.58 129.53
22 129.54 129.56 129.54
23 129.52 129.56 129.57
24 129.50 129.56 129.54
25 129.50 129.57 129.52
26 129.50 129.57 129.51
27 129.54 129.55 129.51
 

hlsmith

Less is more. Stay pure. Stay poor.
#2
you can just end-up with a mean and standard deviation. You would need a gold standard measuring device or the true measurements to get at accuracy. Now you are just looking at variablility or reliability.
 
#3
But in this case I would have 27 different means and 27 standard deviations, is there a tool that could indicate how reliable the whole measurement would be?

Thanks.
 

rogojel

TS Contributor
#4
Hi,
if you google Measurement System Analysis you will find a lot of relevant information. It is basically a two way ANOVA (if you check different operators as well) and you can find a good implementation of it in Minitab. I ran your data through Minitab assuming only one operator and it gave a very good result saying that 0,55% of the total variance in your data set is attributable to measurement noise the rest is due to variation in the parts (positions) you measured. If you need details on MSA I would be happy to help.

regards
 
#6
Good afternoon,

I tried to do the two-way ANOVA analysis using excel. I attached two files: One showing how I organized the data in excel - Two operators (Guilherme and Connor); and 27 different locations of the part measured (despite the picture shows until 8), where 4 measurements were made at each location - and the other file showing the analysis that excel pulled out for me. So, from what I understood if the null hypotesis is true my measurement device presents precision. And F values < Fcrit, and P>0.05 (for 95% confidence) would indicate that teh null hypotesis is true. So, from my second file attached, could I conclude that my device is messed up?

Please, if anyone could correct me I would be glad, as I an a beginner in statistics.
 

Miner

TS Contributor
#7
As rogojel indicated, this is a type of measurement system analysis (MSA). Specifically, in the MSA jargon, it is called repeatability, which is the measurement variation seen by a single operator measuring the same subject multiple times. It assumes that the subject is not affected by the measurement.

The repeatability of your measurement device is s = 0.0228
 
#8
Sorry about that, I think my attachments are bad. Follow again:





Miner, thanks for the response. did you get this value from my attachments or working with the values in my first post?
 

Miner

TS Contributor
#9
Your attachments were too small for me to read. This was from my own calculation using Minitab's Gage R&R Study. However, you can obtain the same result by performing a 1-way ANOVA using the 27 measurement locations as the levels. The pooled standard deviation was 0.023
 
#10
Miner, I cannot see a way to perform a one way ANOVA as I am working with two non-numeric variables (the operators and the differents parts of the piece) and one numeric variable (the measurements itself). I tried to attach an excel file now to replace my old attachments. Lets see if it is going to work.
 
#12
I took the data from the first post as an example. It consists of old data pulled from the same device. The data in the excel spreadsheet I attached now was measured this week specifically to conduct a statistical study, and reorganized in a way to be conducted a two-way ANOVA.
 

rogojel

TS Contributor
#13
hi,
I ran the analysis of your data, first with Minitabs MSA tool which gave the resuts below, without the operator-part interaction


Zweifache ANOVA-Tabelle ohne Wechselwirkungen

Quelle DF SS MS F p
Part 7 8169,11 1167,02 850,320 0,000
Operator 1 511,89 511,89 372,978 0,000
Wiederholbarkeit 55 75,48 1,37
Gesamt 63 8756,48


This is the simple ANOVA analysis and shows that the parts and the operators both influence the final data set. To gage the magnitude of the influence we have here below a more detailed analysis:


R&R (gesamt)

%Beitrag
(der
Quelle VarKomp VarKomp)
R&R (gesamt) 17,326 10,63
Wiederholbarkeit 1,372 0,84
Reproduzierbarkeit 15,954 9,79
Operator 15,954 9,79
Zwischen den Teilen 145,705 89,37
Gesamtstreuung 163,032 100,00


This gives the components of Variation in absolute values and percentages of the total variance. One can see that about 10% of the total variance is comming from the measurement noise (called R%R total (gesamt in german) and 9.8 % is coming from the operators measuring slightly differently, 0,84% from the variation due to the method. So, in total, quite a good method, might need a bit of operator training to make it better.


the same info can be gathered if I just run a Balanced Anova, need only to make sure that I will get the components of variation:

ANOVA: Mes vs. Part; Operator

Faktor Typ Stufen Werte
Part Zufall 8 1; 2; 3; 4; 5; 6; 7; 8
Operator Zufall 2 C; G


Varianzanalyse für Mes

Quelle DF SS MS F p
Part 7 8169,1 1167,0 850,32 0,000
Operator 1 511,9 511,9 372,98 0,000
Fehler 55 75,5 1,4
Gesamt 63 8756,5


S = 1,17151 R-Qd = 99,14% R-Qd(kor) = 99,01%



Again the parts and the operators are significant influences.

The importance of the influence comes from the components of vaiation again:

Erwartetes Mittel der
Quadrate für jeden Term
(unter Verwendung des
Quelle Varianzkomponente Fehlerterm uneinschränkten Modells)
1 Part 145,705 3 (3) + 8 (1)
2 Operator 15,954 3 (3) + 32 (2)
3 Fehler 1,372 (3)


We do not get the percentages, but that is not a problem. The numbers are pretty much the same, what the ANOVA calls Error is considered the measurement system noise in the MSA output.

regards
rogojel
 

Miner

TS Contributor
#14
I attached the results of a Minitab analysis performed two different ways. The first uses the Gage R&R analysis and the second uses a Two-way ANOVA.

One caution about the Gage R&R Analysis is in the percentages. % Contribution is not an issue as it use the variances, which are additive, to calculate the percentages. However, % Study Variation uses standard deviations, which are not additive, to calculate percentages. This results in inflated percentages, which are overly conservative as pointed out by Dr. Donald Wheeler. This is a carryover from early automotive, possibly Mil-STD days, so there is little traction to correct it.
 
#15
So, from rogojel analysis I have got 9.8% of the variance due to reproducibility and 0.84% of the variance due to repeatability.

From Miner Gage R&R I have got 2.59% of the variance due to those two factors and16.09% of the standard deviation. The number of distinct categories also seems right, 8, I believe we are looking for a value higher than 5. The second analysis, Minitab two way ANOVA, I was not able to interpret.

I am going to try to run this data in minitab, but I believe that the general idea is that the system is good but can be improved in terms of repeatability, right?

You both helped me starting understand about the interpretation of data, Thanks! Please correct me if I said something wrong or missed some important information.
 

Miner

TS Contributor
#16
Actually, the area in most need of improvement is Reproducibility. The variation due to Reproducibility is much greater than that of Repeatability.


  • The Operator results in the two way ANOVA matches the Operator results in the Gage R&R ANOVA table.
  • The Repeatability line of the Gage R&R table corresponds to the Error term in the two-way ANOVA.
  • The Interaction line in the two-way ANOVA corresponds to the Operator x Location term in the Gage R&R ANOVA table.
Both use the same math, the results are simply displayed differently.