Comparing groups of male and female participants for a survey (without random sample)

[RamonNL]

Hello everybody and thanks for reading

PROBLEM:
A survey was sent out to the male and female participants of a charity-ran program implemented in China, Congo, and Egypt. Among others, the survey monitored whether participants had a positive impression of the program (with possible answers: yes or no). The answer shows the following percentages of satisfied participants:

Females
China 65% (297 satisfied / 457 total)
Congo 94% (17 satisfied / 18 total)
Egypt 77% (89 satisfied / 115 total)

Males
China 47% (78 satisfied / 167 total)
Congo 78% (46 satisfied / 59 total)
Egypt 73% (401 satisfied / 551 total)

QUESTION:
What is a good way to compare differences across proportions of satisfied participants for the two sexes per country?
And across the two sexes for all countries (considering the different number of participants across countries)?

DISCUSSION:
A statistical test (t or z scores) to compare sets of proportions of male % vs. female % in China, for example would be pointless in my view because the survey was not initially delivered to participants drawn from a random sample of the participant population. Useless to conduct such tests in this case then, right? If my assumption is correct, are there any other tests aside from t and z scores that do not require a random sample and I could use, or any other approaches you could recommend (ratios, weighted average of percentage, or other tests)?

 

[Karabiner]

You have a binary dependent variable, therefore t-test or z-test cannot be used.

But neither for t-tests nor for tests for binary variables random sampling is
absolutely necessary, as long as the observations are independent. The
difficulty (IMO) lies in the precise definition of the population to which you can
generalize your results.

With kind regards

Karabiner

 

[RamonNL]

You have a binary dependent variable, therefore t-test or z-test cannot be used.

But neither for t-tests nor for tests for binary variables random sampling is
absolutely necessary, as long as the observations are independent. The
difficulty (IMO) lies in the precise definition of the population to which you can
generalize your results.

With kind regards

Karabiner

Hi Karabiner

Many thanks for your message. I would like to get back to your point about random sampling not being absolutely necessary for t tests or z tests in this case. Do you have any literature / sources to share? The Statistics books I've checked claim that using a t test to compare two independent population proportions (my case) requires my data to come from a random sample from the population of interest. This is essential for inferential Statistics. I've always thought t and z tests can only be used in this sense and for this purpose.

On your second point about the use of binary variables, I agree logistic regression could be applied.

I also think both tests would be fine. Males and Females can be thought of a binary variable, but also categorical. Males and females can also be thougth of as two independent groups and, because of this, I think one could compare them as such without turning them into a binary variable.

Your comment on the definition of the population is important for the analysis, good heads-up. It is only the survey participants but here again, the absence of sampling is, for me, a major issue.

Have a good weekend,
Ramon

 

[Karabiner]

Many thanks for your message. I would like to get back to your point about random sampling not being absolutely necessary for t tests or z tests in this case. Do you have any literature / sources to share? The Statistics books I've checked claim that using a t test to compare two independent population proportions (my case) requires my data to come from a random sample from the population of interest. This is essential for inferential Statistics. I've always thought t and z tests can only be used in this sense and for this purpose.

I cannot remember having ever seen a single study which claimed that its observations were based on a strictly random
sample of the population of interest, so I 'm afraid that I cannot share any references.

With kind regards

Karabiner

 

[RamonNL]

I cannot remember having ever seen a single study which claimed that its observations were based on a strictly random
sample of the population of interest, so I 'm afraid that I cannot share any references.

With kind regards

Karabiner

Agreed, sometimes I have seen studies that rely on stratified, cluster samples, among others. Thanks for your comments.

Kind regards
Ramon

 

[noetsi]

I do random samples at work. I don't of course publish them. Its hard to believe it does not occur.

There are rules about sampling that do not apply when cluster sampling is used. It is harder to generalize correctly with a stratified sample. Random sampling is much better for accurate results.

 

[RamonNL]

I do random samples at work. I don't of course publish them. Its hard to believe it does not occur.

There are rules about sampling that do not apply when cluster sampling is used. It is harder to generalize correctly with a stratified sample. Random sampling is much better for accurate results.

I also use random samples in 99% of cases, and as a very last resort consider the alternatives, with reluctance of course.

Find it hard to accept some participants will have a greater chance to be selected than others, it is just really unfair

 

[noetsi]

The real issue is generally your sampling frame. You can't be selected if they don't know you exist.