Understanding the significance of my logistic regression model

#1
Hi, I'm a doctor and I'm relatively inexperienced with logistic regression but I'm trying to do it for a project.

I have a binary dependent variable (problem occurs or doesn't occur) and am trying to identify risk factors for the problem occurring. Multivariate logistic regression seems to be a standard method for doing this in medicine. I identified the candidate independent variables based on clinical hypotheses and data from other groups. When I plug in the 5 categorical variables and 2 continuous variables in the "binary logistic" function in SPSS, both the model and the variables are non-significant. If I delete variables, the model and one of the variables becomes more significant, but when I pare the model down to 3 variables, the overall significance of the model is still only 0.055. The total number of cases is 120.

Can someone help me with this? It's fairly common in medical papers to have a logistic regression model in which most of the variables are insignificant and only a few are significant. Are those models just not significant overall? Am I not using this method correctly?
 

hlsmith

Not a robit
#2
You have 120 cases, but what is the breakdown (proportions) between the two outcome groups? What do you plan to do with these results? If you plan to disseminate, you would call it multiple logistic regression not multivariate logistic. Also, feel free to post your model output, so we can see what you are writing about and glean additional details like how many groups are in your categorical variables.
 
#3
Hi Jennifer.

Additionally

If you have a good theoretical reason to include a variable in the model you shouldn't exclude it only because it is statistically insignificant.

a significant level of 0.05 is the maximum type 1 error allowed (rejecting a correct H0). I don't really see a big difference between a p-value of 0.045 and 0.055, but you need to put the line before doing the experiment. I know that in medicine it is usually 0.05.
So back to the test power
 

ondansetron

TS Contributor
#4
I agree with mostly everything posted in reply.

@hlsmith is right to point out that multivariable and multivariate are very different things though they sound the same and the terms are used incorrectly quite frequently in medicine (multivariable is the case of a single outcome modeled by some relationship with many predictors/covariates). This is similar to how non medical people often say "flex" when referring to any muscle contraction or joint movement where a medical person would know that a muscle contraction can lead to flexion at a joint or extension (but that layman will use "flex" for any kind of intentional contraction of muscle).

@obh has a good point about letting theory guide selection of variables when present. If something is theoretically important it isn't unreasonable to enter the variable into the model and leave it without checking a significance test. Also agree with the sentiment that .055 and .045 aren't very different from one another but medicine other fields have an idea that there is some inherent "truthiness" about .05. Just as a patient's sodium can technically be "abnormal" at 130 but the clinician isn't very concerned (due to subject matter expertise and understanding of the clinical picture), so too can you use reasonable judgement in the interpretation of a p-value that is close to a predetermined significance threshold. This isn't saying to "game" anything, but the point is that the p-value is really a continuous measure of evidence against a hypothesis, so close numbers don't necessarily mean different things when they're on different sides of an imposed cutoff (especially when your subject matter knowledge leads you to have suspicion about something in particular). Observational studies are subject to many sources of confounding and excess variation that may zap power or bias estimates so less strong conclusions are reasonable (i.e. maybe noting there is weak evidence of a relationship between that variable and the outcome, but a future study properly designed to investigate this relationship is warranted).

We're happy you're on here and asking questions, so feel free to ask for clarification on anything! If you post output and answer some questions as @hlsmith said, we will take a look!