Leave-one-out cross validation

#1
Hi, I am trying to understand the Leave-one-out cross validation method but all sources explain it mathematically.
Can someone please basically explain how this technique works without using maths?
Thanks
 

hlsmith

Omega Contributor
#2
LOO, is a validation method, which runs your procedure over and over again traditionally dropping a single observation every time. So if you were running a logistic regression with a sample with 20 observations, then the program would run the same model 20 times with n=19. So it would run it excluding the first observation, then the second, then the 3rd,..., then the 20th.
 
#3
LOO, is a validation method, which runs your procedure over and over again traditionally dropping a single observation every time. So if you were running a logistic regression with a sample with 20 observations, then the program would run the same model 20 times with n=19. So it would run it excluding the first observation, then the second, then the 3rd,..., then the 20th.

Thanks. What is the advbantage of doing this? Why repeat the process so many times?
 

bryangoodrich

Probably A Mammal
#4
Thanks. What is the advbantage of doing this? Why repeat the process so many times?
To avoid overfitting a model to data. If you fit a model to all 20 data points, it will try to do its best to fit the model (so structured) to the data. Great, but what about when you apply that model to *new data.* That is where out-of-sample testing (cross validation) comes in handy.

By using LOOCV, you're fitting the structure (model) 20 different times and testing how well it predicts that out-of-sample data point. If it does well compared to other structured models, it is better at matching what that data represents. In other words, you're not over fitting the model to its training or in-sample data.

Other forms of this sort of CV is k-fold (typically k = 5 or 10) where instead of leaving one out (k=20 in this example), you break the data into k groups, fit to k-1 groups and predict the kth one held out, repeat k-times. By averaging the error observed over the k out-of-sample tests, you get a better measure of how the model will handle data of the sort it is trained on. It better generalizes to new data. It's a better predictive model we might say.
 

hlsmith

Omega Contributor
#5
An advantage is that you don't need a large dataset as in k-fold CV. Since you are not splitting your data up into larger proportion sets but but running the n - 1 models.