+ Reply to Thread
Results 1 to 10 of 10

Thread: level of significance 0.1%

  1. #1
    Points: 1,186, Level: 18
    Level completed: 86%, Points required for next Level: 14

    Posts
    3
    Thanks
    0
    Thanked 0 Times in 0 Posts

    level of significance 0.1%




    Hi!

    Would it be OK to use a level of significance of 0.1 -instead of traditional 0.05- when using non-parametric tests in bivariate analyses if clinical significance is suspected? If so, could you please give me some references of papers supporting this assumption?

    Thanks a lot!!

  2. #2
    Points: 2,993, Level: 33
    Level completed: 62%, Points required for next Level: 57

    Posts
    87
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: level of significance 0.1%

    I don't personally know of any references supporting this idea at all. You would need to really go out of your way, in my opinion, to justify the use of this threshold, parametric or otherwise.

  3. #3
    Super Moderator
    Points: 31,766, Level: 100
    Level completed: 0%, Points required for next Level: 0
    bugman's Avatar
    Posts
    2,255
    Thanks
    290
    Thanked 324 Times in 265 Posts

    Re: level of significance 0.1%

    I'm not sure you would need to really go out of your way to justify this.

    0.05 is not set in stone as ,however, it is the generally accepted and expected alpha value in most disciplines.
    As long as you justify your choice with sensible reasoning, you should be fine - and backing this up with a peer reviewed article wouldn't hurt.

    I have used 0.1 in the past, when, for environmental impact statements we want a more "relaxed" threshold to act as an early warning signal between reference and impact sites (as an example).
    The earth is round: P<0.05

  4. #4
    Points: 2,993, Level: 33
    Level completed: 62%, Points required for next Level: 57

    Posts
    87
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: level of significance 0.1%

    I suppose coming from a behavioral sciences I may not have an accurate understanding of acceptable practices in other realms such as environmental sciences.

  5. #5
    Points: 2,513, Level: 30
    Level completed: 43%, Points required for next Level: 87

    Posts
    194
    Thanks
    1
    Thanked 14 Times in 13 Posts

    Re: level of significance 0.1%

    I agree with bugman that there is no reason you should not use confidence threshold different from 0.05, as long as you say what threshold you are using.

    But what I can't see how to justify is using one confidence threshold for parametric tests and another for non-parametric tests. There is nothing fundamentally different about what confidence means for a parametric vs. non-parametric test.

    Pick your desired confidence interval and pick the appropriate test, but those are independent decisions based on entirely different criteria.

  6. #6
    Points: 1,186, Level: 18
    Level completed: 86%, Points required for next Level: 14

    Posts
    3
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: level of significance 0.1%

    Thanks a lot for your quick answers!!

  7. #7
    Points: 2, Level: 1
    Level completed: 3%, Points required for next Level: 48

    Posts
    1
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: level of significance 0.1%

    Hi Juanpe000,

    I am facing the same problem that u did, so did u manage to find any reference on that? Really thanks a lot.

  8. #8
    TS Contributor
    Points: 14,811, Level: 78
    Level completed: 91%, Points required for next Level: 39
    Miner's Avatar
    Location
    Greater Milwaukee area
    Posts
    1,171
    Thanks
    34
    Thanked 405 Times in 363 Posts

    Re: level of significance 0.1%

    It is really a matter of risk. If you are performing exploratory studies with plans for future confirmation studies AND the risk associated with making a Type 1 error is low then 0.10 is an acceptable alpha. If on the other hand, the risk associated with a type 1 error is very high (e.g., potential for injury/harm) then 0.01 might be more appropriate.

    Alpha should always be established based on risk, not because it is "traditional" or generally accepted.

  9. #9
    TS Contributor
    Points: 18,889, Level: 87
    Level completed: 8%, Points required for next Level: 461
    CowboyBear's Avatar
    Location
    New Zealand
    Posts
    2,062
    Thanks
    121
    Thanked 427 Times in 328 Posts

    Re: level of significance 0.1%

    Some serious thread necromancy going on here, but I'll take the opportunity for a couple of choice quotes:

    Quote Originally Posted by Rosnow & Rosenthal, 1989
    Surely, God loves the .06 as nearly much as the .05 (link)
    (I'm not sure how God feels about the .1 though)

    Quote Originally Posted by Cohen, 1990
    The null hypothesis, taken literally (and that's the only way you can take it in formal hypothesis testing), is always false in the real world. (link)
    So the marginal probability of a Type I error in the real world* is 0.

    That all said, while the probability of a Type I error may be 0, the probability that a reviewer will spit the dummy at the prospect of alpha = 0.1 is approximately 1.

    *May depend on your definition of real world.

  10. #10
    TS Contributor
    Points: 17,742, Level: 84
    Level completed: 79%, Points required for next Level: 108
    Karabiner's Avatar
    Location
    FC Schalke 04, Germany
    Posts
    2,539
    Thanks
    56
    Thanked 639 Times in 601 Posts

    Re: level of significance 0.1%


    So the marginal probability of a Type I error in the real world* is 0.
    Yes, but often we don't know the direction of the effect for sure.

    With kind regards

    K.

+ Reply to Thread

           




Similar Threads

  1. level of significance
    By slm0530 in forum Statistics
    Replies: 0
    Last Post: 05-06-2008, 12:58 PM
  2. 0.05 Level of Significance... HW HELP!
    By ik066287 in forum Statistics
    Replies: 0
    Last Post: 12-16-2007, 03:52 PM
  3. Significance Level
    By nat9 in forum R
    Replies: 1
    Last Post: 10-21-2006, 09:53 AM
  4. .05 level of significance?!!!
    By michelecm in forum Statistics
    Replies: 0
    Last Post: 07-06-2006, 02:12 PM
  5. What exactly is the p-value and the significance level?
    By garfunkel41 in forum Statistics
    Replies: 1
    Last Post: 04-27-2006, 03:14 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats