Overall Rating for Doctors Based on IVR Data

[HanHin]

Good Day,

I have IVR "Interactive Voice Response" data from Health care system, where patient can give their opinion about the service they receive from the doctor.
the are no criteria; it is only one question they have to answer which is "Are you satisfied about the service you received ? "
Response choices are:
1: Not Satisfied
2: Not Quiet Satisfied
3: Neutral
4: Satisfied
5: Very Satisfied

I have got some doctors with loads of patients that their responses were between 1-2; and other doctors with few patients that their responses were 4-5.
What I need to do with the data is to give overall rating to the doctors.
I would like to take into consideration the fact that a pressure on the doctor to serve as many patients as they could and long waiting time are factors to receiving low satisfactions.

I need to find a fair way to give a proper rating.

I would appreciate your help.

Thank you


Hanan

 

[hlsmith]

There are no other applicable items on this 'survey', say wait time?

 

[Karabiner]

Maybe you can include some variables who probably have an influence on rating,
regardless of the doctor's quality. You identified workload/limited time per patient.
Are there additional factors which you can use?

By the way, how many patients and how many physicians are included?

With kind regards

Karabiner

 

[HanHin]

There are no other applicable items on this 'survey', say wait time?

Maybe you can include some variables who probably have an influence on rating,
regardless of the doctor's quality. You identified workload/limited time per patient.
Are there additional factors which you can use?

By the way, how many patients and how many physicians are included?

With kind regards

Karabiner

No, it is only one question, there are no other Qs such as:
what is the visit type? follow-up or first visit? because the maximum time for follow-ups is 10 min and for first visit is 15 min per patient
how long did they wait to see the doctor?

I only have the "evaluation time" which is the date and time when the survey was taken which doesn't indicate nothing.

The reason they say: they don't want to overwhelm the patients.

Someone told me that I should give the patient a value so " when the patients counts increased the patient value gets less" but it did not make sense to me.

Thank you so much

Hanan

 

[Karabiner]

Could you tell us how many doctors there were, how many patients in total, and
what the range was for number of patients per doctor?
Were survey about a doctor taken on only one day, or were they taken across
a longer time span?

I only have the "evaluation time" which is the date and time when the
survey was taken which doesn't indicate nothing.

Did you already test this? Maybe there are seasonal effects.

With kind regards

Karabiner

 

[HanHin]

Could you tell us how many doctors there were, how many patients in total,

The IVR data gets batched every month. So I cannot be sure if the call was made on the same day of the patient's visit or after.

what the range was for number of patients per doctor?

I cannot really tell, because it depends on the doctors' preferences and how do they make their appointments schedule,
but number of patients based on the recommended time of visit types would be:
24 patients for follow-ups, and
16 patients for new visits.

Were survey about a doctor taken on only one day, or were they taken across
a longer time span?

on one day for out-patients.
for in-patients after they get discharged, so it is a time span.

Did you already test this? Maybe there are seasonal effects.

I don't follow, what do you mean?


Thank you very much, I appreciate you help


Hanan

 

[Karabiner]

Ok, if you only want to take into account workload/number of patients:

You could make an X-Y-scatterplot for number of patients per doctor & mean evaluation of a
doctor, in order to check whether there is an association. And you can do a simple linear regression
of mean rating on number of patients and use the regression weight as correcting
factor, by lookin at the residals from that regression, i.e. by looking at the difference between each
doctor's actual mean rating and the mean rating which would be expected due to his workload.

Another approach:
You could make a histogram for "number of patients per doctor" (as proxy for waiting time) and
decide whether ist could be justified to make 3 (or perhaps more) groups, i.e. something like
"doctors with the lowest 33% of number of patients", "doctors with the medium 33% number of
patients", doctors with the largest 33%. Within each of this groups, you can rank doctors (percentage
ranking, for example), so that you can say from each doctor in each group if he's among the best or
among the not-so-good doctors.

With kind regards

Karabiner

 

[HanHin]

Thank you so much, It is very helpful.

 

[HanHin]

Hello Karabiner,

I went with the first approach. I did the scatterplot to check if there is a relation between number of patients and low evaluation.
Here is the scatterplot chart.



What is your opinion, please?

Thank you,

Hanan

 

[Karabiner]

Ok, so there's a non-linear association. That makes the first approach a bit complicated.
One would have to logarithmically transform both variables, I suppose
Is the doctor with 60 patients a valid value?

We can see that if a doctor has 20+ patents, he cannot receive much more than 1.5 points,
on average. So I'd be inclined to say, make some groups based on number of patients, and
assign each doctor a rank within his group.

With kind regards

Karabiner

 

[HanHin]

Is the doctor with 60 patients a valid value?

Karabiner

What do you mean? Yes, I checked the raw data.

So I'd be inclined to say, make some groups based on number of patients, and
assign each doctor a rank within his group.

Karabiner

The minimum number of patients is 1. Number of patients goes like this 1-11, 13-17, 20, 22-23, 26-28, 30, 33, 60.
What would be the right approach to divide them into groups?

Thank you,
I appreciate your help,


Regards,

Hanan

 

[Karabiner]

There is no right approach, only approaches which are more or less defendable.
If you got information about what the typical categories of workload are (something
like "1-5 is low", "6-15" is medium, > 15 is high") you could this.

A crude approach would be categorizing b percentiles, i.e. lowest/medium/highest
33% (or maybe 4 quartiles) of workloads.