+ Reply to Thread
Results 1 to 4 of 4

Thread: Checking significance of bias.

  1. #1
    Points: 689, Level: 13
    Level completed: 78%, Points required for next Level: 11

    Posts
    11
    Thanks
    3
    Thanked 0 Times in 0 Posts

    Checking significance of bias.




    In a new design, estimator seems to be biased but efficient than the existing design. Now I want to check whether the bias of the estimator based on new design is significant.

    The formula of the variance of the estimator based on new design is intractable.

    How can I check the significance of bias of the estimator based on new design?

  2. #2
    Omega Contributor
    Points: 38,374, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,998
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: Checking significance of bias.

    Use simulated data, so true effects are known and thencompare error estimates.


    P.S., Bootstrap may be your friend here.
    Stop cowardice, ban guns!

  3. The Following User Says Thank You to hlsmith For This Useful Post:

    user1234 (07-24-2017)

  4. #3
    Points: 689, Level: 13
    Level completed: 78%, Points required for next Level: 11

    Posts
    11
    Thanks
    3
    Thanked 0 Times in 0 Posts

    Re: Checking significance of bias.

    Could you please give me a toy example in R so that I can understand how is to check significance of bias?

  5. #4
    Omega Contributor
    Points: 38,374, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,998
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: Checking significance of bias.


    Not a wizard in R or in simulation. But you would create the true data generating process:


    P(X1=1) = 0.6
    P(X2=1) = 0.4
    P(Y = 1 | X1, X2)


    Then fit the traditional approach and your novel approach and get estimates. Depending on your model (e.g., linear, logistic, etc) you get your error parameter (e.g., RMSE, accuracy, etc.) as well. To further put it into perspective, you can get 95% percentile confidence intervals via bootstrapping. Next overlay the two histograms. You can also mark on the figure what the true effect was using a reference line. This approach would contrast the results from the two model specifications.


    One thing to note is that your results would only be for one possible data generating process, simulation. Meaning the bias may be different for other data generating processes, say when other covariate distributions are used, interaction terms, etc. This is a general heuristic approach for intractable problems I believe. It can help in trying to discern the direction and magnitude of bias.
    Stop cowardice, ban guns!

  6. The Following User Says Thank You to hlsmith For This Useful Post:

    user1234 (07-24-2017)

+ Reply to Thread

           




Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats