+ Reply to Thread
Results 1 to 4 of 4

Thread: Performance of CSIs at Crime Scenes

  1. #1
    Points: 12, Level: 1
    Level completed: 23%, Points required for next Level: 38

    Posts
    2
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Performance of CSIs at Crime Scenes




    Hi,

    Im after some advice, which i would really appreciate assistance with.

    At our police force the CSIs are being performance tested by use of statistics. The statistics reveal the following (per 3 months):

    (1) How many fingerprint/DNA matches the CSI has gained (linking the crime scene sample to the offender)
    (2) The number of crime scenes attended
    (3) The number of exhibits seized


    Then calculated is
    (1) The average fingerprint/DNA identifications for the office/individual - then each CSIs identifications are compared to the office average.
    (2) The percentage of fingerprint/DNA identifications from the total number of exhibits seized


    The problem I have with these statistics is that
    (1) Not all crime scene are the same, there are so many vairables, one might attend serval burglary scenes and find glove marks left at the scene
    (2) There is an assumption that fingerprint/DNA identifications are equally spread across every CSI in the office.
    (3) Not all crime scenes render evidence - shed/garage burglary will have little to nothing evidentially, wheras a burglary may have much more due to the nature of the incident
    (4) The jobs we attend are not equally distributed, we are dispatched to jobs as, and when they become available, and what scene your given is not your choice.

    I think there are several more 'assumptions' associated with these calculations. Id be grateful is you could advise on your views please.


    This is a new thing which is being done at the police force which i have a problem with, it just doesnt seem fair to me, as there are so many variables.

    We are trying to writeup a statement to management to say that these stats are not the way forward. The reason im writing here is if you can assist in sharing your stats knowledge which may give us more ammunition to show how skewed the stats calculations are.

    Thank you

  2. #2
    TS Contributor
    Points: 12,227, Level: 72
    Level completed: 45%, Points required for next Level: 223
    rogojel's Avatar
    Location
    I work in Europe, live in Hungary
    Posts
    1,470
    Thanks
    160
    Thanked 332 Times in 312 Posts

    Re: Performance of CSIs at Crime Scenes

    hi,
    there us a deeper problem with the method as you explained it . We sometimes define performance measures for companies and the one warning / question we always give them is: would it be good for the company if people changed their behavior in a way to optimize these measures?

    E.g if I were measured on the number of identifications, I would definitely avoid difficult cases like the pest.

    My suggestion would then be to think about the behavior changes this measurement would bring about and argue from that angle .

    disclaimer This is definitely not to say that measurements are bad, they just need to be very carefully designed.

    good luck
    rogojel

  3. #3
    Points: 12, Level: 1
    Level completed: 23%, Points required for next Level: 38

    Posts
    2
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Performance of CSIs at Crime Scenes

    Quote Originally Posted by rogojel View Post
    hi,
    there us a deeper problem with the method as you explained it . We sometimes define performance measures for companies and the one warning / question we always give them is: would it be good for the company if people changed their behavior in a way to optimize these measures?

    E.g if I were measured on the number of identifications, I would definitely avoid difficult cases like the pest.

    My suggestion would then be to think about the behavior changes this measurement would bring about and argue from that angle .

    disclaimer This is definitely not to say that measurements are bad, they just need to be very carefully designed.

    good luck
    rogojel

    Thank you so much for your reply.

    I really appreciate your suggestion, it was something I didnt think about but is a very valid point and a way to approach the problem.

    I would like to approach the statistical side of things too, from looking at the points I made in the original post - is there any points I need to make to show that the statistics are wrong, invalid and flawed to judge the performance of a CSI? - I assume you work in the field os statistics so your views will be very, very much appreciated.

    Thanks again

  4. #4
    TS Contributor
    Points: 12,227, Level: 72
    Level completed: 45%, Points required for next Level: 223
    rogojel's Avatar
    Location
    I work in Europe, live in Hungary
    Posts
    1,470
    Thanks
    160
    Thanked 332 Times in 312 Posts

    Re: Performance of CSIs at Crime Scenes


    Hi,
    I am a bit wary about going into the details because I have no practical knowledge of the field, but maybe a few more thoughts:

    - another question to ask would be imho the goal of the measurement, the way this data is intended to be used. E.g. if this is to rank the people - it should be noted that at any given time roughly 50% of the force will be below average (a well known statistics joke in fact). For ranking purposes all the confounding factors you listed, and more, will be active - so the simple average is a singularly ill chosen metrics for that. In fact, given the possibilities to game the system I would just not use any measurement for ranking.

    In the book "Thinking like a Freak" there is some good advice on this:

    1. Create incentives that switch the frame from adversarial to cooperative
    2. Never ever think people will do something just because it is the right thing to do
    3. Know that some people will do everything to game the system finding ways to win that you never could have imagined.

    - if the goal is to gain an overall view of the units performance then it would make a lot of sense to anonymize the measurements (as no one is interested in individual performance, right?) and use the right type of control chart . It would have the advantage of providing the information without the incentive to game the system. Just to gain more insight I would gather the factors that influence the metrics (e.g. a rating of the site on a Likert scale (very Easy, easy, average, Difficult , Very difficult) and record that information and evrything else you come up with, too.

    So, I would really discuss this from the angle of cooperation - can the measurements be used in a way to help the employees? Are they used that way? If yes on both counts, then you can (will) go into the discussion of how to frame them so that their efficiency is maximized.

    I know this is not exactly what you expect, but maybe it will help you a bit?

    regards
    rogojel

+ Reply to Thread

           




Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats