+ Reply to Thread
Results 1 to 12 of 12

Thread: Frequent ED attenders - statistical test

  1. #1
    Points: 8,411, Level: 61
    Level completed: 87%, Points required for next Level: 39

    Posts
    20
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Frequent ED attenders - statistical test




    Hi I've a problem at work and looking for some suggestions as to how to progress.

    We took a sample of 100 patients. For each patient we calculated the average number of times they attended the hospital's Emergency Department over a period of time. (These patients were known to be frequent attenders).

    Following a project designed to reduce ED attendances we counted the same patient's attendance again over a period of months and calculated the mean average.

    Knowing that patients will regress to their means over time, the frequency of attendance naturally decreasing and increasing, what test will show if the campaign was successful and any reduction is not simply regression to the mean (RtM) for each patient.

    I was thinking of a binomial test. If a patient attended 8 times in 10 months, giving an average of 0.8, what is the probability they would attend 4 times in 12 months after. If the p < 0.05 we can discount RtM. Unfortunately though some patients have a monthly average of 2 or 3 and it does not work.

  2. #2
    Omega Contributor
    Points: 38,334, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,998
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: Frequent ED attenders - statistical test

    Quick design question, what was the mode of transportation to the hospital for these patients and how can you be sure they just didn't go to another hospital during the first time period or second (more often). Do you know the denominator of total ED visits.


    Was the intervention a blanket approach to address all patients or was it tailor to these unique patients (such as talking only to them in person about the appropriateness of the visit).


    Also, have you thought about treating these as count data?
    Stop cowardice, ban guns!

  3. #3
    Points: 5,083, Level: 45
    Level completed: 67%, Points required for next Level: 67

    Posts
    109
    Thanks
    19
    Thanked 3 Times in 3 Posts

    Re: Frequent ED attenders - statistical test

    When you say regress to their mean, do you mean that you expect the patient's attendances to regress to the old mean (that which you calculated before the intervention) or do you expect the mean attendances to regress to a new, hopefully lower, mean (presumably because of the intervention?). In other words, do you expect the intervention only to have a temporary affect on the reattendances?

  4. #4
    Omega Contributor
    Points: 38,334, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,998
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: Frequent ED attenders - statistical test

    I think they were trying to in their mind account for extreme cases for patients seen the first month. A patient that had 10 visits which usually has 0; a patient with 1 visit which usually has 10. Not sure if you should really be concerned about this, though think about the following:


    much depends on the appropriateness of these visits. Was it deemed these are inappropriate ED utilizations?
    Stop cowardice, ban guns!

  5. #5
    Points: 5,083, Level: 45
    Level completed: 67%, Points required for next Level: 67

    Posts
    109
    Thanks
    19
    Thanked 3 Times in 3 Posts

    Re: Frequent ED attenders - statistical test

    My experience of repeat attenders (i worked in ED for 5 years as a nurse) was that they would have a flourish of attendances within a period of time, followed by a barren spell.

    Generally repeat attendances are considered inappropriate, unless the attendances are clinically unrelated.

  6. #6
    Points: 8,411, Level: 61
    Level completed: 87%, Points required for next Level: 39

    Posts
    20
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Frequent ED attenders - statistical test

    Wow thanks for the help.

    The project is geared only to frequent users and is not a blanket campaign and you're right Prometheus, the behaviour of frequent users is erratic. For this reason I have data 12 months before intervention and 12 months post. Regarding regression to the mean, I'm looking at the average attendance before intervention.

    I'm starting to think about treating these as counts hlsmith and looking up Poisson tests for means. My thinking is that if there is no sig. difference in means and especially if the variance is smaller after intervention, it is just regression to the mean. However if the mean attendance is sig. lower then irrespective of variance we can discount regression to the mean.

    As for attendance at other hospitals, we don't have the data but it is something we have considered.

    Andrew

  7. #7
    Omega Contributor
    Points: 38,334, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,998
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: Frequent ED attenders - statistical test

    I am sure you are looking at quite a bit, but this question is riddled with many issues. Things to think about:


    how many visits do you miss when they are seen at another ED pre and post (how do you know the true denominator of total visits)
    are you just scaring them to the other ED if they are direct admissions
    if they shouldn't have come to the ED, was a clinic open or was this their only option
    can you control for their overall health
    how are you defining if they should have came to the ED or not - are you calling an encounter appropriate or not. One period could be heavy on inappropriate encounters and the other with appropriate encounters.
    Are you controlling for support (marital status, etc.)
    Does their condition affect their mentation
    you have not control group - can you find patients you did not intervene on and see if their rates changed (you would want to match or use propensity scores).
    if patients came in multiple times did they get multiple exposures to the intervention
    what if a patient died or was admitted to skilled care then they are not coming back as much what if a patient was lost-to-follow (or ability to come back, moved)
    is it the patient's choice if they call EMS (or are the facility selection out of their hands (regardless of appropriateness, given medics transport them)
    When they come in is their a cyclical antecedent that may be the driving force that may or may not be annual or seasonal.
    Does the change in more uninsured people having potential access to converge in the US interrupt either of your time periods. Could the person lost coverage over the time period so the intervention did not balk usage but insurance status.



    Big picture, how do you know you actually made a change if you only collect data from your facility (are they opting not to come in or just going else where) with no control group with randomization of intervention.


    Sorry for the messiness of my flow of though!
    Stop cowardice, ban guns!

  8. #8
    Points: 8,411, Level: 61
    Level completed: 87%, Points required for next Level: 39

    Posts
    20
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Frequent ED attenders - statistical test

    Thanks for the informative reply hlsmith and I agree and over time we intend to consider the points you raised. I have already raised some of them in meetings. That said, I work in the UK so there is no insurance problem. 90% of patients used the ambulance service so going to another hospital is unlikely though not entirely out of the question, but from passed behaviour it would be doubtful given the way our ambulance service works. The patients all live in an area serviced by the hospital in question. The ambulance would not take them elsewhere even if asked.

    This project has been running for little over a year (though I was only brought in last week). It required some investment from the hospital and before it invests any more to keep the project running wants to see evidence that the reduction is a true reduction and not simply the natural variation in attendance figures. Many of these patients were brought into the program during a peak period of activity so naturally we would expect the activity to decline. Their past behaviour shows this; there are peaks with lots of attendances followed by quiet periods with zero attendances.

    The million dollar question is, was the reduction greater than what would be normally expected if there was no intervention? Which I interpret as regression to the mean. Moods median (the data is skewed) and a Wilcox test show p < 0.05, but is that sufficient to discount RtM? Also to understand the cost savings they want the pre-intervention data modelling so that the difference between actual and modelled activity can be costed and the savings realised.

    That's the bit I'm struggling with. I only have counts per patient for each of the twelve months pre and post intervention. For example for one patient I have, 1,2,0,0,5,4,0,1,0,1,6,7 for pre and a similar set for post. This data would give the patient an average pre-intervention attendance over the twelve months of 2.25. If post intervention the data looked like 4,0,1,0,0,0,3,0,0,1,0,0 we have a post mean of 0.75. A Poisson means test is p<0.05 and the confidence intervals do not overlap. Can I take it that this is not RtM?

    However if the post attendance counts were 4,3,0,5,1,0,0,3,3,0,1,0 we have a post mean of 1.7 and a p value of 0.38. Also the variance is lower than the pre-intervention data and so one could conclude that this is RtM and most likely not due to intervention.

    How you then model the pre-intervention data to approximate RtM to calculate the cost saving beyond what would otherwise be expected is a whole other problem.

    I also have of course gender, age and medical condition (alcohol, self harm, loneliness etc) and they will be used later but for the moment the emphasis is on the number of attendances to determine if there is a true return of investment for the hospital and not just random variation giving the illusion of a saving. The last thing the accountants want is to invest a mighty sum only to find attendance rising again the next year because in reality there was no improvement.

    Andrew
    Last edited by scotchpie; 08-26-2015 at 03:05 PM.

  9. #9
    Omega Contributor
    Points: 38,334, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,998
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: Frequent ED attenders - statistical test

    Andrew,


    Adding confidence intervals is always good (for the sampling variation) and noting the change in variance is a good things as well. I guess I am familiar with the topic of RtM, but unsure how it would be addressed in this scenario. If I get time I may investigate.


    If you run a poison regression, you can throw those other covariates right in the model and perhaps also look at any potential interactions between terms.


    Side note, I have never used it, but there is also something called discontinuity regression, where you compare some people right near the threshold of getting the intervention to those who got the intervention but were right near the threshold. This is done to compare similar samples with or without the intervention.
    Stop cowardice, ban guns!

  10. #10
    TS Contributor
    Points: 14,811, Level: 78
    Level completed: 91%, Points required for next Level: 39
    Miner's Avatar
    Location
    Greater Milwaukee area
    Posts
    1,171
    Thanks
    34
    Thanked 405 Times in 363 Posts

    Re: Frequent ED attenders - statistical test

    You might also consider a nonparametric recurrent event analysis (e.g., mean cumulative function, Nelson-Aalen plot).


    See http://support.minitab.com/en-us/min...on-aalen-plot/ and http://www.weibull.com/hotwire/issue57/relbasics57.htm


    Don't get thrown off by the references to reliability or failures. It is simply a recurring event in time.

  11. The Following User Says Thank You to Miner For This Useful Post:

    Prometheus (08-27-2015)

  12. #11
    Points: 5,083, Level: 45
    Level completed: 67%, Points required for next Level: 67

    Posts
    109
    Thanks
    19
    Thanked 3 Times in 3 Posts

    Re: Frequent ED attenders - statistical test

    Hey Miner, that's some interesting looking stuff (if only i could understand it!). Just at quick glance it looks a little familiar, does it have any relation to survival analysis?

  13. #12
    TS Contributor
    Points: 14,811, Level: 78
    Level completed: 91%, Points required for next Level: 39
    Miner's Avatar
    Location
    Greater Milwaukee area
    Posts
    1,171
    Thanks
    34
    Thanked 405 Times in 363 Posts

    Re: Frequent ED attenders - statistical test


    Very similar. Survival analysis is for non-recurring events (per subject), while these are for recurring events (per subject). The distinction is that non-recurring events are independent of each other while recurring events are dependent.

+ Reply to Thread

           




Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats