How do you know the measurements are off? Is there a validation variable collected. If you know the bias, there may be a sensitivity analysis to incorporate.
What is this variance ratio you mentioned?
hi,
just had a scary thought . in all industrial DoEs we definitely have measurement error on the IVs. However I have always seen the OLS as the proposed method to analyse them. Is this an error? Should we actually use a Deming regression or some more sophisticated method?
If the variance of the X measurements is less then the variance of the Y measurement this might be ok, but I have never seen this comparison be actually performed or even proposed.
regards
How do you know the measurements are off? Is there a validation variable collected. If you know the bias, there may be a sensitivity analysis to incorporate.
What is this variance ratio you mentioned?
Stop cowardice, ban guns!
Hi,
they are generally not off pe se but have their own measurement variance so I habe no way to know whether the level thatbwas set is the level actually achieved. This should be incorporated in the analysis - actually I can not see an application of multiple fegression where this would not be the case.
Well issues are typically selection bias, measurement error, confounding, and model misclassification. Your concerns are that there may be bias? I can remember what Deming reg entails. Would it help.
I think you need to think about if there is systematic error such as a pattern in the mismeasure or if it was random.
Stop cowardice, ban guns!
If all you want to know about is the relationship between the X measurements and Y, then there is no need to correct for measurement error. We need only to assume that future X measurements will contain about as much error as the current X measurements. But if your interest is in the relationship between Y and the underlying X "true scores"--that is, the underlying thing supposedly being measured--then correcting for measurement error would be appropriate. In science we are more often interested in the latter, but in industrial experiments you may well only care about the former.
In God we trust. All others must bring data.
~W. Edwards Deming
Taking on the psychometric perspective of measurement error, I see...
...
...
well, then I guess I can't really add anything else, LOL.
But I am curious, though... is there any reason as far as why you would make a distinction between, say the process of measurement and (statistical) analysis in science VS in industry? I mean, as a business owner I could see myself interested in making sure I can measure things (sales, amount of product distributed, etc.) as accurately as I can.
for all your psychometric needs! https://psychometroscar.wordpress.com/about/
So you are not able to measure with the exactitude fully desired. I guess it comes down to your purpose and if there is any concern toward systematic error. What are your thoughts?
Stop cowardice, ban guns!
Hi,
I could imagine several scenarios here : e.g. I could measure X precisely at a given moment but I can not keep it constant during the experiment - imagine it is a reaction temperature that is regulated by a thermostat for instance. I set the value and the "real "temperature will go up and down according to the algorithm the thermostat follows. So my original X is just a rough approximation of what I actually do - but I would consider it a fixed given value in the OLS regression.
The other alternative would be a measurement that really has a lot of variation - like measuring viscosity for instance. My "real" input could be anywhere in a wide range of possible values due to measurement variation (it is not a bias BTW) I just do not know the real value. Again, the traditional DoE analysis with OLS would just assume the X has the value we measured.
Of course, in both cases we would be interested in the relationship between the real X and the real Y, the measured /set values being more or less random.
When I don't know the answer to a good question, like rogojels, I try to google and search for it.
This is what I found, a thesis by Angela Mcmahan "Measurement Error in Designed Experiments for Second Order Models".
The first 4 - 5 pages seems to summarize about 'classical measurement errors' and 'Berksom errors' and properties for OLS estimates.
I could see real world temperatures always being variable given where it was taken and ambient characteristics. It isn't measurement error given the variable being recorded, but I still think it falls under bias, because it is not a recording of the true measure of the underlying phenomena. We just don't like hearing the word bias and our minds cringe and associate it with systematic errors. But if I want to know the temp on something given an experiment (equation) and I can't not control for everything my errors result in a bias. Think about this under the guise of the bias/variance tradeoff.
I digress. The big thing is that I believe this always happens in experiments, but most move forward with standard approaches and just document these limitation in their write-up. I am off to check out GG's link.
Stop cowardice, ban guns!
rogojel,
You work in manufacturing, correct? What follows, is correct for industrial statistics. I'm not certain whether it can generalize to other branches of statistics.
In industrial statistics, DOEs are used for 4 purposes:In both Screening and Robust Optimizing, you are typically looking for moderate to large effects and the errors in measurements (EIM) issue is too small to worry about. If they were large enough to impact the results and decisions made, you have other problems to worry about and the effects are too small to be of practical value.
- Screening: to identify the few significant factors that affect a process
- Modeling: to create a mathematical model describing the relationship between process factors and outputs
- Optimizing: to use the model to minimize/maximize/target the process output
- Robust optimizing (desensitizing): to make the process output less sensitive to variation in the factors
For Modeling and Optimizing, the EIM issue might cause a practical problem if your model needs to be very precise. In my personal experience, this issue was never a problem in the manufacturing environment. However, I have seen several product design applications where this could have caused some practical issues. These were situations where the model needed to be very precise and was used by the product's control software.
Tweet |