Machine Learning Fundamentals: Cross Validation
One of the fundamental concepts in machine learning is Cross Validation. It's how we decide which machine learning method would be best for our dataset. Check out the video to find out how!
For a complete index of all the StatQuest videos, check out:
statquest.org/video-index/
If you'd like to support StatQuest, please consider...
Buying The StatQuest Illustrated Guide to Machine Learning!!!
PDF - statquest.gumroad.com/l/wvtmc
Paperback - www.amazon.com/dp/B09ZCKR4H6
Kindle eBook - www.amazon.com/dp/B09ZG79HXC
Patreon: / statquest
...or...
KZhead Membership: / @statquest
...a cool StatQuest t-shirt or sweatshirt:
shop.spreadshirt.com/statques...
...buying one or two of my songs (or go large and get a whole album!)
joshuastarmer.bandcamp.com/
...or just donating to StatQuest!
www.paypal.me/statquest
0:00 Awesome song and introduction
0:25 Motivation for using Cross Validation
1:18 Cross Validation concepts
3:41 An example using Cross Validation
4:35 Terminology (4-Fold, 10-Fold, etc)
5:20 Cross Validation for tuning parameters
Correction:
4:16 KNN should have 10 correct and 14 incorrect.
#statquest #ML #crossvalidation
Correction: 4:16 KNN should have 10 correct and 14 incorrect. NOTE: There has been a debate if we should call the "testing dataset" a "testing dataset" or "validation dataset". In my opinion, this depends on the size of your dataset. We'd all like to have a large dataset that we can divide into three parts: Training, Validation and Testing, but that doesn't always happen in the real world. Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/
lol I stopped at that point for one minute wondering why it is 10 and 12 for which the sum is not 24
tiny bam!!
Can you clarify what is this "Correct" and "Incorrect" indicating after each testing using different blocks of data?..what is the interpretation when correct:4 ? :( Unable to get it.. :(
@@ahanapal4055 The machine learning methods that I am comparing in this video are classifying observations. Since we are training the methods, we know how the observations should be classified in advance. Thus, if the method makes the correct classification, then it is "correct". If the method makes the incorrect classification, then it is "incorrect". Does that make sense?
@@statquest yes, thanks a lot for the clarification!!
My friends find me lame when I say "I learn machine learning from a guy who sings and teaches" . Lol they are missing out.
That's funny. :)
Tiny Bam!
yup joshuastarmer.bandcamp.com/track/love-song
I think he is got the world's best teaching skills. Trust me learning ML is not easy unless you are interested. Even if you are not at least you will not feel sleepy in his lectures.
Angry bam ! 😤
It’s crazy to think where we would be if every subject had videos this clear and well made.
Thanks!
its crazy to think where i would be if i j had access to the net in my growing years instead of my abusive dad
we can't control that assholes brought us into the world but thank goodness we have videos now to get us where we need to be
@@khushiiversee I can understand your feelings. All I could say is to let you know you are not alone.
This guy is legend better than top university professors 😆
Thanks!
@@statquest As a student at a top 10 uni in the world, I can confirm these are facts 😂
@@j.castro7355really? 😂 Then i will no longer have an excuse that i don't have access to the best education anymore. Let's grind hard
Couldn’t agree more. After going through machine learning course materials on virtually every educational platform like coursera, simplilearn, EdX from top universities and companies from Harvard to Google, I think none of them remotely reaches the clarity and no-bushitness here. BAM!!!!
Facts!
Dear sir/ Dear Josh, Your StatQuest series is brilliant to say the least. The internet is these days flooded with ML tutorials that teach how to run algorithms such as logistic regression or KNN using softwares, or with the lengthy incomprehensible mathematics that explains those algorithms. Yours is one of the rare materials that explains the philosophy! Philosophy, that is the deal for humans, not just feeding numbers and generating more numbers using a machine. Thanks a lot for giving me clarity on how exactly to use cross validation, and for clearing some of the nagging doubts from my tiny,less intelligent brain .
Hooray! I'm glad you like my video. :)
yes, that relates to me very much. I'm now in a Data Science bootcamp, and they just explain the maths behind each of algorithms incomprehensibly. they said that the point is just know the little math, because on the field, we just import the sklearn library, and try every model, every algorithm, which one gives the best prediction.... after listening to their statements like that, it makes me wondering. "hmm, i'm afraid that they are probably true, that there is no point at all to learn the math behind these ML Algorithms, because they just import module, choose each of existing algorithm, and done"
This is the awesome video. TRIPLE BAM!!!!!
Hooray!!! :)
What's the difference between a machine learning method and machine learning model? Is a model applying a method to a specific dataset therefore modeling how it behaves? Will you make a statquest about what is a model??? That's a triple question mark bam! Love your videos! Thank you!
@@jcourn1 do you still need an answer or should I just skip it, since you posted it an year ago.
@@yashasvibhatt1951 thanks for replying! Sure! What's the implication of the terms machine learning model?
@@jcourn1 A Machine Learning Method is a way of teaching a machine using the data-driven approach. A Machine Learning Algorithm is a set of rules or a list of steps or a procedure to teach the machine using that methodology. A Machine Learning model is what we have received after applying the algorithm on a certain dataset to teach our machine. It represents what was learned by machine using the algorithm. Hope that helps 🙂🙂🙂
i like how you're trying to make your videos not only educational but also entertaining
Thanks!
Why can't most other lecturers on this world teach like you, why can't MY lecturers teach like you, im crying now :(((( if I have to learn Stats/AI/DL/... every single day for the rest of my life, but if it's you who taught us, it's well worth it.
Thanks!
probably some of the best explained stats videos i've seen on youtube. thank you josh for constantly providing us with material that we can actually understand 0:)
Thank you very much. :)
If I pass my machine learning exam next week it will literally be all thanks to you. Either my book is completely unreadable or I'm stupid, but your videos make so much sense and I finally feel like I actually get the stuff you're talking about. Thank u!!
Thanks! By the way, I have a book covering this same material - so check it out if you need extra help: statquest.org/statquest-store/
@@statquest I PASSED!!! THANK YOU SO MUCH!!!! :D
@@profetspurvius913 Congratulations!!! TRIPLE BAM!!!
The best example so far. After watching this, my lecture's notes made sense.
Same case with me. Double Bam! :)
How'd the rest of your class go? Was it a...Bam!?
I can read books and listen to professors for hours about a subject like this and still not understand it... then I watch a 6 minute video and it is crystal clear. Thank you StatQuest!!!!!!!!
bam! :)
Josh Starmer, you are the savior of my PhD! I rarely do this, but I'm gonna buy a shirt... THANK YOU!
Hooray! And thank you very much! :)
I cannot believe how you put all these complicated theories into such an explicit way!!! Wonderful channel!
Thank you very much! :)
my teacher took almost 2 hours to explain this and i didn't even get it! THANK YOU I got it in under 10 Minutes !!
Glad it helped!
there are a lot of teachers that have knowledge , but of them 80 percent dont know how to teach , 10 percent knows but dont care, 8 percent really care but are not succinct with their methods but 2 percent knows how to teach clearly and precisely in layman terms , they can teach anyone with their style , You are in that 2 percent category . Respect >>>>>.
Thank you very much! :)
I hardly ever comment on KZhead videos, but I just wanted to say that this has been a TREMENDOUS help and I absolutely loved the breakdown, logic, humor, and visuals. Thank you for for making this brilliant video!
Thank you!
NO words for you Mr Josh, hats off!! You make all the concepts so easy to learn in such a short time.
Thank you!
Thank you very much for this video, Josh! The use of visuals to explain cross validation really helps! I learnt a lot through this video about the fundamental basis behind cross validation as well as the extreme case of Leave-One-Out!
Hooray!
I have completed Applied Machine Learning course from a University in US. The concepts I learned there are being reinforced after watching your Video Josh. Thank you so much for putting out these videos.
!!! BAM !!! Finally I found a KZhead trainer who shares knowledge the way I would like to learn... A big thank you :)
Hooray! :)
Hey Josh! This is the first time I'm watching your videos and I love the way you teach: pausing for a second before saying the next sentence. It gives time for the listener to digest what you said before! Love it!
Awesome! Thank you!
I just don't understand how somebody could dislike this video. It has everything I've ever wanted teaching to be.
bam! :)
Who also liked the video upon hearing the short musical interlude at the beginning?! Your voice is very soothing. I’m preparing for an exam in about 2hours and needed to understand this concept. Thanks a lot! Every single info in this video came in my exams! I wrote with understanding!!! Thanks!
Thank you very much! :)
I have a ML quiz on monday and was so worried about not grasping these concepts in time - your videos are super clear and helpful and genuinely enjoyable to watch! Thank you StatQuest with Josh Starmer
Hooray!!! Happy to help.
I can't imagine how my life would be without these videos! Thanks a lot!
Hooray! I'm glad the videos are helpful. :)
Dude your explanations and visuals are just perfect. I will watch each and every video uploaded by you for sure.
Thank you very much! :)
I loved the Tiny Bam, Sir! You Patience to go slow tell us that you have a low bias; meaning, it's easy for non-native English folks to understand the concepts clearly. Keep up the good work. I will stalk your channel and like all the videos you have every made by the end of this week. Thank you Again.
BAM! :)
Firstly i like to thank you for explaining these concepts in such a crystal clear manner , this is one of the best video i ever witnessed. second, i request you to please make some video on backpropagation and some tedious concepts of M.L. once again thank you.
Oh my sweet lord! I couldn't have ever imagined that someone can teach data science concepts soooooooo interestingly and easily. I never ever comment!! But you made me do this first time in my life
Thank you very much! :)
Best concept descriptions I have found yet. Explained over-fitting in a better way that my textbook or course have. Hoping for a linear algebra course! Thanks!
Awesome, thank you!
Your clips really save me from my lack of basic knowledge and fear of machine learning. Horay........... Triple Bam.!!!!
Glad to help!
Actually, it is an exciting stat quest!! Thanks so much for your videos. Concise, informative and entertaining. Much appreciated.
This is fantastic, I usually don't comment, but felt I had to from how well done this explanation is. Thank you for taking the time to make this
Thank you so much! :)
These videos do such an amazing job summarizing concepts that my professor has spend hours trying to explain. I was pulling my hair in frustration at his teaching until I encountered your videos. These videos are like a breath of fresh air to my knowledge and understanding of data science. A huge thanks to you Josh Starmer! Keep up the amazing work!
Glad to help!
The amount of BS they try to get you to wade through when explaining concepts E.g. Instead of starting with a massive equation and the formal explaination, a simple intuitive explainatiom, then relate that to the formal process
I finally understand machine learning and it's better explained than in class. You're the best, BAM!
Happy to help!
When I am gonna make my videos, you'll be my inspiration. The way you take us through the video is like a guide taking us through a guided meditation. Edit : and at the end it make us feel satisfied and delightful.
Wow, thank you!
This is the best stat channel. Extremely simple to understand. Thank you!!!
Thank you! :)
Clearly explained, great video! Maybe you skip this on purpose due to its complexity, but there is a small caveat. At the end you mention 'parameter tuning' using cv, these 'parameters' are called hyperparameters, different as model parameters. In order to do so, you need to further split the data into train/validation/test set, and only use train/validation part for tuning, while still having the test set for a final estimation of model performance.
please , I have question regarding cv for ridge regression , I will try different (lamnda) in each fold for example (10 different values for lamnda ) with ten folds or should I try each (lamnda) I need to test with all 10 fold and compare in the final between them
Just saying I've watched only 3-4 of your videos and you have me hooked! Best, concise and simple explanation!
Wow, thanks!
I just can't explain how much i love your teaching!!! the songs refreshen my mind every time...
Hooray! :)
I'm so glad I have found your channel, extremely well explained and I was in a good mood from the start because of the epic song.
Awesome, thank you!
your channel is the best channel I've seen in KZhead!!! Look forward for more videos!!!
Thank you so much!!! :)
would you like to talk about cost complexity pruning when you have time? thank you!
I struggled with the concept for a bit, it became instantly clear to me! Thanks a lot.
Awesome!
Amazing! Explains basic concepts very well, wish I had seen this video when I had no clue about training/testing etc.
Better late than never. :)
I find this crazy that before and after every (very expensive) class now I'm looking up the same info here.... I'm a top-down learner though and my class seems to be built around bottom up learners. Thank you soooo much - yes I'll get a hoodie! #statquestforlyfe
BAM! I'm glad the videos are helpful! :)
BAAAAAAAM!!! That was awesome expression. Wish you had practical examples worked on MATLAB or Phyton.
That clarity I get after watching your videos... ! BIG BAM!
Thank you!
Your videos are very helpful, much practical and simple way to explain concepts. I learned more in your videos than my grad lecture notes. Thank you so much!
Awesome! :)
the easiest video on the Internet to understand this topic :)
What you do is simply amazing!!!!! Thank you!!!! Just a tiny question: when you divide your data into blocks which you use as training set, do you use each different block for a different algorithm, or do you use the same training data to train different algorithms? Thank you again!
If you split your data into blocks 1, 2, and 3, then you would train all of your models on blocks 1 and 2 and test with 3. Then you would train all of your models on blocks 1 and 3 and test with 2 and then you would train all of your models on blocks 2 and 3 test with 1. bam.
@@statquest Great Josh!!! Thank you very much! This channel is a life saver!
One of the most wholesome channels on here; absolutely love it, I'm getting motivated instantly !
Thank you!
Oh my god this guy rocks! Clearest explanations for understanding this stuff, by far!
some scientists should take example as you just explain , congratulations JOSH !
Thanks! :)
ᵗⁱⁿʸ ᵇᵃᵐ
Perfect!
Your teaching style is just awesome. You explained everything in simple words and great English accent which is easily understandable. You got a new subscriber
Thank you! 😃
first time watching your video for an exam, really felt the BAM moment! 👏 you're a wonderful teacher, please keep up the good work!
Thank you! 😃
Given the 1000th like to this video :)
Thank you Josh! As always, it's super fun to watch and learn with your videos.
Hooray! :)
Before watched Statquest videos, I NEVER believe I'd like numbers, statistics... Now I believe I could be a data scientist if I keep watching these videos:-)
bam! :)
Am I the only one who thinks that show casing 2 talents at same time is becoming new phenomenon?
lol at tiny bam
:)
Loool.
one of the best videos ever i have watched, made machine learning clear only in 1.17 min of the video, you man are very great
thank you very much! :)
Thanks josh you are the best source for understanding the intuition behind every concept in Statistics and Machine Learning.
Thank you!
Thank you for the video! Do we need to perform a loss function?
The machine learning method you use might involve a loss function, but, otherwise, you don't need to use one.
BAM that subscribe button!
Double BAM!!! Thank you!
Good vid, this is k fold cross validation, the notion of a cross validation set involves dividing your data even further for hyper parameter tuning.
Dude I came to understand the difference between Cross Validation and Leave one out, instead I found that i completly missunderstood cross validation. Happy that i had a big breakthrough, i decided to watch the video to the end. And DOUBLE BAM in one sentence you explained what leave on out is. -> Subscribed!
Hooray! I'm glad video was helpful.
I watched 50% for ML and 50% for the BAMS!!
BAM! :)
Finally, this youtube algorithm take me here.
Bam! :)
this Guy is a G. Just found his channel. one of the Best series of lectures out there. Thanks.
Thank you!!! :)
bought 3 of your albums today. I'm a big fan! keep up this awesome channel!!
TRIPLE BAM! Thank you very much! :)
This shit is legit.
Thank you for simplifying cross-validation concepts. It helps me a ton for my masters. Again, thank you!
Glad it was helpful!
New subsciber here, I can't believe I'm late to this channel. THANK YOU SO MUCH. You have explained it in the clearest way possible!
Thank you very much! :)
You're the best teacher ever! Your videos motivate me not to give up in Data Science!! Thanks a lot!!
Thanks! I'm glad they're helpful.
I use a tenfold cross-validation method in the ridge and lasso regression implementation in my master thesis on SONAR/RADAR imaging. At that time I read a lot about Cross-validation to grasp the concept. Today your video help me to brush up the concept again. Thanks a lot. and feel bad that time I did not found this channel.
I'm glad the video was helpful! :)
I have been struggling with this concept but you cleared it within 6 mins wow thank you!!!
Hooray! :)
Thank you Josh Starmer for your excellent work, I personally enjoy watching your tutorials.
Thank you!
It's almost 3am and I need to go to sleep, but I can't stop watching your videos! They are awesome. Thank you so much!
BAM! :)
Directly went to BAM.. no need to even think about SAM :D Referring to samtools here..
Just as I was getting seriously over my head with K Fold CV for a Numerai model... Lo and behold! My favorite statistical troubadour, Josh, appears to light the way. Bam to every which way you can validate it!
BAM! :)
You're amazing. Thanks a lot. With statquest, machine learning is child's play. Thanks Josh, your efforts very much appreciated
Thank you! :)
by far the best video explaining this concept. Thank you!!!
You're welcome! I'm glad you like it. :)
My face hurts from smiling so much at these! Thanks so much! Your videos are so helpful for me to understand my new job!
BAM! And congratulations on the new job. :)
Thank you. BOMBASTIC BAM. It's super easy to comprehend. Now I'm gonna share this video like crazy!! BAM BAM BAM
Awesome!!! Thank you very much.
You are an AWESOME Teacher !! Thanks a lot for making Machine Learning very easy for us !!
Thank you! :)
All the things are crystal clear, you are doing a very good job, you are amazing man....hats off.
Thank you so much 😀
Great work Josh, really great explanations and content being explained. thank you
Thank you! :)
SUPER BAM!! Awesome explanation and a wonderful sense of presentation to go hand in hand!!
Thank you very much! :)
Really great video ! Concepts are explained with many confidence. Keep it up !!!
Thanks!
What an easy-to-understand explanation! Thank you!
Thanks!
Nice and super clear. Best explanation for cross validation.
Thanks! :)
Josh, what is your own way of learning new things? Your ability to simplify things so well shows that you have a deep understanding of the subject.
I just read everything I can about a topic and then re-read and re-read and re-read until I learn. The trick is that I never give up.
I watched your every video and you have great sense of teaching. Thanks
Thank you! :)
1:18 ML methods, Logistic regression, K nearest neighbours, Support vector machines. Cross validation allows us to compare different ML methods and get a sense of how well they will work in practise. We need two things to do with the data collected. i) estimate the parameters for machine learning method(training the machine learning method) ii) test the machine learning method(evaluation of the model) 4 fold cross validation,leave one out cross validation, 10 fold cross validation(commonly used), tuning parameter
double bam!
Josh your songs are so relaxing, it is like I´m stressed out, then let´s watch a StatQuest, they always start with those funny songs.
Hooray!! Thank you very much.
BAM! Thank you very much for this valuable piece of content. Cross validation is as clear as water to me now.
bam!
Brilliant! Loved it! Thank you so much for simplifying that, my Friend!
My pleasure!
Words fail me. Mr. Starmer, you have a true gift for teaching. If you are ever in Amsterdam, the drinks are on me.....
Hooray! Thank you very much! :)
Love how he teaches us like we are 6 year olds haha. You earned yourself a subscriber. Thank you for your videos!
Thanks!