+ Reply to Thread
Results 1 to 5 of 5

Thread: Leave-one-out cross validation

  1. #1
    Points: 166, Level: 3
    Level completed: 32%, Points required for next Level: 34

    Posts
    6
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Leave-one-out cross validation




    Hi, I am trying to understand the Leave-one-out cross validation method but all sources explain it mathematically.
    Can someone please basically explain how this technique works without using maths?
    Thanks

  2. #2
    Omega Contributor
    Points: 38,374, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,998
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: Leave-one-out cross validation

    LOO, is a validation method, which runs your procedure over and over again traditionally dropping a single observation every time. So if you were running a logistic regression with a sample with 20 observations, then the program would run the same model 20 times with n=19. So it would run it excluding the first observation, then the second, then the 3rd,..., then the 20th.
    Stop cowardice, ban guns!

  3. #3
    Points: 166, Level: 3
    Level completed: 32%, Points required for next Level: 34

    Posts
    6
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Leave-one-out cross validation

    Quote Originally Posted by hlsmith View Post
    LOO, is a validation method, which runs your procedure over and over again traditionally dropping a single observation every time. So if you were running a logistic regression with a sample with 20 observations, then the program would run the same model 20 times with n=19. So it would run it excluding the first observation, then the second, then the 3rd,..., then the 20th.

    Thanks. What is the advbantage of doing this? Why repeat the process so many times?

  4. #4
    Probably A Mammal
    Points: 31,087, Level: 100
    Level completed: 0%, Points required for next Level: 0
    bryangoodrich's Avatar
    Location
    Sacramento, California, United States
    Posts
    2,564
    Thanks
    398
    Thanked 618 Times in 551 Posts

    Re: Leave-one-out cross validation

    Quote Originally Posted by jsmith613 View Post
    Thanks. What is the advbantage of doing this? Why repeat the process so many times?
    To avoid overfitting a model to data. If you fit a model to all 20 data points, it will try to do its best to fit the model (so structured) to the data. Great, but what about when you apply that model to *new data.* That is where out-of-sample testing (cross validation) comes in handy.

    By using LOOCV, you're fitting the structure (model) 20 different times and testing how well it predicts that out-of-sample data point. If it does well compared to other structured models, it is better at matching what that data represents. In other words, you're not over fitting the model to its training or in-sample data.

    Other forms of this sort of CV is k-fold (typically k = 5 or 10) where instead of leaving one out (k=20 in this example), you break the data into k groups, fit to k-1 groups and predict the kth one held out, repeat k-times. By averaging the error observed over the k out-of-sample tests, you get a better measure of how the model will handle data of the sort it is trained on. It better generalizes to new data. It's a better predictive model we might say.
    You should definitely use jQuery. It's really great and does all things.

  5. The Following 2 Users Say Thank You to bryangoodrich For This Useful Post:

    bugman (02-01-2016), gianmarco (02-02-2016)

  6. #5
    Omega Contributor
    Points: 38,374, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,998
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: Leave-one-out cross validation


    An advantage is that you don't need a large dataset as in k-fold CV. Since you are not splitting your data up into larger proportion sets but but running the n - 1 models.
    Stop cowardice, ban guns!

+ Reply to Thread

           




Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats