Controlling for response bias where all goals are positively related to well-being: regression or mean-centering?

[Frank Magique]

Hi,

I am conducting a study where I've asked participants to rate how much they aspire towards 13 different life goals and want to know how upholding these various goals is related to well-being. However, it has been noted in the literature - and my data shows this as well - that essentially all goals are positively related to well-being. Thus, one needs to correct for this response bias. Instead of zero-order correlations, the key question is whether prioritizing certain goals over other goals is related to well-being/ill-being.

Two approaches have been used in the literature to examine this:
1) Regression analysis where the importance of goal in question and average goal importance (calculated by taking the average of all studied goals) are simultaneously regressed on well-being. The overall goal importance thus acts here as a control variable, and one asks whether the individual goal importance has any direct relation to well-being beyond the relation explained by overall goal importance.
2) Mean-centering where each goal is mean-centered to the participant’s mean rating of overall goal importance, by subtracting the mean across all aspirations from each aspiration. This way one gets variables that show how much more (or less) the participant strives for a certain goal in contrast to how much one strives for goals overall. Then one examines these new variables' relation to well-being.

I now have to choose which analytic approach to use to control for response bias. And I have been asked by the editor to justify my choice. Thus I am trying to figure out what are the different theoretical assumptions and implications between these two approaches.

Anyone who could weigh in and tell under what assumptions and in what situations one should prefer one over the other?

 

[kiton]

Thank you for a good question. Although I am not an expert in response bias, here some of my thoughts about this. Response bias could be driven by the following: people are inclined to say “yes”, people act differently when they are taking part in research, some people tend to use extreme ends of the scale, people want to appear like they are “good people”, different people interpret scales differently. Furthermore, it is noted that regardless of the different styles of response bias, people are not usually trying to lie or mislead the researcher. Therefore, it seems to me that it is a peculiarity of this specific theory on goals and well being that the relationship is positively correlated. As such, if your goal is not to prove the existence of the bias, then it shouldn't raise a significant concern (unless perhaps you observe a negative correlation).

Now, turning to your empirical approach. I believe the answer is in your descriptions of those -- whatever goal suits your research question, then that would be appropriate to use. Personally, I would pick one that satisfies my RQ and then run a robustness test using another one (assuming it is still relevant to what I aim to study).

 

[Frank Magique]

Thanks kiton for your reply! I think the reason all goals tend to relate positively to well-being is that people higher on well-being tend to be more goal-oriented (contra a depressed person who on average probably would score quite low on all goal pursuits). So that might explain the positive zero-order correlations. So what one is controlling is the overall level of goal striving in order to see what effect on well-being does prioritizing one goal over the others have.

...But I feel that both regression where one controls for overall goal striving and mean-centering can achieve that aim, and thus I still don't have a good rationale to pick one over the other.