KZhead

The KL Divergence : Data Science Basics

ritvikmath
2024 ж. 25 Мам.
38 907 Рет қаралды
Жүктеу

understanding how to measure the difference between two distributions
Proof that KL Divergence is non-negative : • Jensen's Inequality : ...
My Patreon : www.patreon.com/user?u=49277905
0:00 How to Learn Math
1:57 Motivation for P(x) / Q(x)
7:21 Motivation for Log
11:43 Motivation for Leading P(x)
15:59 Application to Data Science

Пікірлер

  • Wow... 😳 I've never seen more genius, easy and intuitive explanation of KL-div 😳👏👏👏👏👏 Big thanks good man ! ❤️

    @szymonk.7237@szymonk.7237 Жыл бұрын

    • Glad you liked it!

      @ritvikmath@ritvikmath Жыл бұрын

    • I agree,

      @anilaxsus6376@anilaxsus63769 ай бұрын

  • Your bottom-up (instead of top-down) approach that you mentioned in the beginning of the video would be really great to see for all kinds of differrent concepts!

    @murkyPurple123@murkyPurple123 Жыл бұрын

    • Great idea!

      @ritvikmath@ritvikmath Жыл бұрын

  • I don't think I'm ever going to forget this. Thanks so much.

    @varadpuntambekar8895@varadpuntambekar8895Ай бұрын

  • I am a research scientist. You provide a clear and concise treatment of KL-Divergence. The best I have seen to date. Thanks.

    @zafersahinoglu5913@zafersahinoglu59135 ай бұрын

  • That was the best description of why we use log that I have ever seen. Good work, man.

    @DS-vu5yo@DS-vu5yo9 ай бұрын

  • I'm in the middle of a $2,500 course, BUT → KZhead → your video... 👏🏻👏🏻👏🏻👏🏻👏🏻 Thank you for starting with the "why", and appealing to my brains desire to understand, not just do.

    @user-li5lh1qs6s@user-li5lh1qs6s Жыл бұрын

  • This is mind blowing.... I love the way you go from the problem to the solution, it's clever way to understand this KL divergence

    @trungphan9137@trungphan9137 Жыл бұрын

    • thanks!

      @ritvikmath@ritvikmath Жыл бұрын

  • That was great. I have struggled to understand certain aspects of KL Divergence, and this is a great way to think about it without getting bogged down in symbology.

    @JBoy340a@JBoy340a Жыл бұрын

    • Glad it was helpful!

      @ritvikmath@ritvikmath Жыл бұрын

  • Amazing. The pace you have explained, the approach...everything is just top-notch.

    @tanvirazhar@tanvirazhar10 ай бұрын

  • Blew my mind, I wanted to understand what kl divergence is to understand the recent Gen AI papers and couldn't. This video helped me a lot.

    @RakshithReddy5555@RakshithReddy55555 ай бұрын

  • This video is absolutely mind-blowing! The way it breaks down such a complex concept into an intuitive understanding is truly remarkable. Thank you!

    @asimosman3428@asimosman342810 ай бұрын

  • Thank you for this, the best explanation of KL divergence that I have seen. Love how you approach it building gradually, really inspiring for how to learn math.

    @markozege@markozege Жыл бұрын

  • Wow. This is the best explanation of KL-divergence I've ever heard. So many over-complicated stuff out there but yours is absolutely genius.

    @marka5968@marka5968 Жыл бұрын

    • Glad it was helpful!

      @ritvikmath@ritvikmath Жыл бұрын

  • Let's celebrate a new video on this amazing chanel!!! Love your work!

    @shamarbauyrzhan7997@shamarbauyrzhan7997 Жыл бұрын

    • 🎉

      @ritvikmath@ritvikmath Жыл бұрын

  • wahh.. i am studying computer science master degree. Your video really helps me a lot! please keep on doing such great work for us!

    @tom199520000@tom1995200002 ай бұрын

  • The best explanation I've ever seen about KL divergence ❤

    @godlyradmehr2004@godlyradmehr2004Ай бұрын

  • I think you're channel and teaching style is brilliant. I wish I knew about this channel when I was doing my undergrad.

    @mrcaljoe1@mrcaljoe1 Жыл бұрын

  • Great explanation, this is the first time I'm learning about KL divergence and it was very easy to grasp because of the way you taught it

    @anujadassanayake6202@anujadassanayake6202 Жыл бұрын

  • Best Math Teacher ever. So clearly explained the design and thinking process of how the algo comes out. Many video just explain the formula which confused me why we should do this way... Thank you!

    @eagermage3157@eagermage315711 ай бұрын

  • You've really made my day with ur explanation. Thank you so much :D

    @trentbolt2006@trentbolt20069 ай бұрын

  • In the 'Motivation for log,' you said that taking a simple average is not the right way to go, and then you try to find a function that makes f(4) and f(1/4) have opposite signs. That means you are trying to make two very different distributions have the smallest distance possible (canceling each other out), which is contradictory to what we expected. We expected them to be large.

    @yhoang6674@yhoang667411 ай бұрын

  • That was great! Not just dumping the formula on you but walking you through its logic with simple steps. Loved it! ❤

    @julianwebb9222@julianwebb9222Ай бұрын

  • Great work! I've been a fan of your ,material for some time and in this video you have truly mastered your craft.

    @midnightwanders5876@midnightwanders5876 Жыл бұрын

    • Wow, thank you!

      @ritvikmath@ritvikmath Жыл бұрын

  • the thing you said in the first minute, is something ive been saying for a while now. As students we arent told what problem drove scientists or engineers into constructing new formulas or ways of thinking.

    @andrew-qf4xl@andrew-qf4xl5 ай бұрын

  • I love how you approach to the KL divergence!

    @tampopo_yukki@tampopo_yukki8 ай бұрын

  • mad respect for Ritvik from Ritwik for acing the subtle art of intuitive explanation:)) If only professors could master the same art.

    @jackritwik09@jackritwik095 ай бұрын

  • Outstanding. Really helping me through this info retrieval course!

    @paigecarlson1742@paigecarlson1742Ай бұрын

  • I found out this professor is very good at explaining every tough concept! respect and many appreciations!

    @thankyouthankyou1172@thankyouthankyou11726 ай бұрын

  • superb...I believe this is the best explanation I have ever come across for K L Divergence. Thanks a tonne.

    @somdubey5436@somdubey54364 ай бұрын

  • me not know some of the fundamentals after listening to your explanation made a lot of sense, and I felt I understood the concept well. I am willing to watch your videos more often.

    @user-vb1no5lq1e@user-vb1no5lq1e3 ай бұрын

  • Everytime i have a math question your hannel is my first choice! Amazing ✅ thanks a million 🎉

    @Mars.2024@Mars.20243 ай бұрын

  • Thank you as always for sharing your brilliant teachings, Ritvik. Could you please do a video on the Gram-Schmidt process and how orthonormal basis matrices are relevant to data science?

    @vzinko@vzinko Жыл бұрын

  • Fantastically clearly explained, congrats.

    @andrashorvath2411@andrashorvath2411 Жыл бұрын

  • Your videos are great just keep going, I watched you for few years already

    @momcilomrkaic2214@momcilomrkaic22143 ай бұрын

  • Excellent way to explain it. Makes maths sounds logical and approachable 🎉

    @Hobbies_forkids@Hobbies_forkids Жыл бұрын

  • I recently got interested in learning machine learning and stumbled upon the stable diffusion, the current state of art open source image generation ai. That's where I encountered the KL divergence. The more I try to understand it, more complicated concepts and formulas are thrown at me. I managed to find some videos that explains how to derive it, but none of them explained why the hell logarithm is present in it for gods sake! And here you are, explaining every missing details from other videos and blog posts in a way that the person who knows very little about the subject can understand in a very satisfying and easy to follow way. Hats off to you, sir. I wish every teachers are like you.

    @brandonkim4675@brandonkim4675 Жыл бұрын

    • Thanks and godspeed for your journey through machine learning !

      @ritvikmath@ritvikmath Жыл бұрын

  • One of the BEST tutorials for sure

    @tayyibulhassan6227@tayyibulhassan622710 ай бұрын

  • Another amazing video! Please keep them coming!

    @danscherb4130@danscherb4130 Жыл бұрын

  • Amazing video, love the format!

    @0hexe@0hexe Жыл бұрын

  • This is the perfect video in Math. Love it. Shared with all my readers

    @ChocolateMilkCultLeader@ChocolateMilkCultLeader Жыл бұрын

  • That was one of the best explanations I have ever heard! Great job and many thanks!

    @kasyaci@kasyaci11 ай бұрын

    • Thanks!!

      @ritvikmath@ritvikmath11 ай бұрын

  • The comments didn't lie you actually explained this so well. I watched the ads all the way through btw.

    @SSJVNN@SSJVNN5 ай бұрын

  • Excellent intuitive explanation!

    @seansullivan6986@seansullivan69867 күн бұрын

  • It was the easiest explanation I’ve ever seen.

    @mantische@mantische Жыл бұрын

  • Amazing teaching. It helps a lot in my process of data shift covariate detection project. Thanks

    @Andy-qi5nh@Andy-qi5nh Жыл бұрын

    • Glad it was helpful!

      @ritvikmath@ritvikmath Жыл бұрын

  • Thanks, exactly the explanation I have been looking for!

    @manducchuc915@manducchuc9157 ай бұрын

  • Thank you for the great explanation! I totally agree that math is not given from above, but invented by people. And showing how the invention can be done is the best way to teach the new concepts. Thanks a lot!

    @vorushin@vorushin4 ай бұрын

  • Thank you so much for this explanation and also got a new insight about the log :)

    @PrajwalSingh15@PrajwalSingh15 Жыл бұрын

    • Happy to help!

      @ritvikmath@ritvikmath Жыл бұрын

  • Bro is a legend

    @sunset6109@sunset610910 ай бұрын

  • This was awesome. Really helpful to think through it backwards and “redevelop” our own function

    @winstongraves8321@winstongraves8321 Жыл бұрын

  • dude, the explanation is so good, you rock!

    @Justin-zw1hx@Justin-zw1hx Жыл бұрын

    • Glad it helped!

      @ritvikmath@ritvikmath Жыл бұрын

  • amazing explanation. not many can do this. well done.

    @sandipmehta2950@sandipmehta29509 ай бұрын

  • I have never seen complex math explained this good Thank you very much!

    @mehmetozkan1479@mehmetozkan14797 ай бұрын

  • very nice explanation. Thanks for the work.

    @akhileshpandey123@akhileshpandey123 Жыл бұрын

  • Taking the MITx Stats class, but I find that you explain the concepts so much better!

    @gingerderidder8665@gingerderidder86652 ай бұрын

    • Glad to hear!

      @ritvikmath@ritvikmath2 ай бұрын

  • Awesome explaination. Thanks for this video

    @s.prakash7869@s.prakash7869 Жыл бұрын

    • Glad it was helpful!

      @ritvikmath@ritvikmath Жыл бұрын

  • This was incredibly illustrative!

    @hpp496videos@hpp496videos Жыл бұрын

  • Thank you! This is the best explanation of KL divergence wich i've seen

    @user-co6pu8zv3v@user-co6pu8zv3v Жыл бұрын

    • Glad it was helpful!

      @ritvikmath@ritvikmath Жыл бұрын

  • Thank you. As usual, great and very intuitive explanation.

    @aliksargsyan2035@aliksargsyan2035 Жыл бұрын

    • No problem !

      @ritvikmath@ritvikmath Жыл бұрын

  • This is an amzing explanation, thanks!

    @yb801@yb801 Жыл бұрын

    • Glad it was helpful!

      @ritvikmath@ritvikmath Жыл бұрын

  • That was amazing. Thank you so much!

    @sparkgin@sparkgin19 күн бұрын

  • Thank you SO much! God bless you Sir, keep up the great work 😊

    @s.m.tahsinzaman_2720@s.m.tahsinzaman_2720 Жыл бұрын

    • You are very welcome

      @ritvikmath@ritvikmath Жыл бұрын

  • Great stuff... Learning a way to teach maths to my kid... A constructivist method... While learning about stats... I really appreciate your work.

    @fh3652@fh3652 Жыл бұрын

    • Glad it was helpful!

      @ritvikmath@ritvikmath Жыл бұрын

  • Amazing explanation!

    @aisniper4095@aisniper4095 Жыл бұрын

  • Great video. Two pieces missing for it to be perfect are in my opinion. If you could just calculate for sum of log (p(x)/q(x)) and show us what's wrong with that number. Exactly as you did with simple p(x)/q(x) why it isn't good solution. Finally in last slide if you could give numbers. You tell about quantification of something that is visually clear, but missing the numbers is kinda missed opportunity to explain how it works :) Again - thank you a lot for explanation. Great work.

    @adaslesniak@adaslesniak3 ай бұрын

  • thank you for the clear explanation.

    @keyvan4680@keyvan46806 ай бұрын

  • thank you for the best expanation on this topic

    @barbaraalexandrova6680@barbaraalexandrova66802 күн бұрын

  • Thank you very much for your valuable videos!!

    @ingenierocivilizado728@ingenierocivilizado7282 ай бұрын

    • Glad you like them!

      @ritvikmath@ritvikmath2 ай бұрын

  • Thank you so much for the explanation. it was really helpful👍👍

    @abironnoy3115@abironnoy3115 Жыл бұрын

    • Thanks!

      @ritvikmath@ritvikmath Жыл бұрын

  • It would be interesting to have a video on how you study to understand a topic, what resources you use and the materials you look for

    @TheFirebolt2010@TheFirebolt2010Ай бұрын

  • Math should all be taught this way, and to go one step further we should teach people how to make sense of math themselve in the long run Thanks for the explanation for kl divergence though ;)

    @chaochaisit@chaochaisit Жыл бұрын

  • Awesome! Very intuitive

    @vaibhavnakrani2983@vaibhavnakrani29835 ай бұрын

  • Very good explanation

    @jtkklb@jtkklb24 күн бұрын

  • You just got a subscriber. Thank You! 😊

    @rishi2504@rishi25048 ай бұрын

  • You are great in explaining this! Thanks!

    @sikalee415@sikalee415 Жыл бұрын

    • Glad it was helpful!

      @ritvikmath@ritvikmath Жыл бұрын

  • Wonderful man. Thank you so much.

    @devindoinmonkmode@devindoinmonkmodeАй бұрын

  • Thanks a lot for sharing the underlying motivation behind the K-L divergence! I really needed such deep insights! JAJAKALLAH...

    @komuna5984@komuna598410 ай бұрын

    • You're so welcome!

      @ritvikmath@ritvikmath10 ай бұрын

  • Best video I've seen in a while!

    @razgaon3680@razgaon3680 Жыл бұрын

    • Thanks!

      @ritvikmath@ritvikmath Жыл бұрын

  • Thanks for the lecture, your work is always so intuitive.

    @qiguosun129@qiguosun129 Жыл бұрын

    • You are very welcome

      @ritvikmath@ritvikmath Жыл бұрын

  • God level explanation thank you!!!

    @kaanefe4266@kaanefe42667 ай бұрын

  • Great explanation

    @talsveta@talsveta9 ай бұрын

  • very very very very well explained, Thanks

    @seyyedmahdihoseini3084@seyyedmahdihoseini30846 ай бұрын

  • thank you. this really helped !!

    @afsanarabeya4417@afsanarabeya4417 Жыл бұрын

  • you are way more better than my school's professor. thank you

    @ringo8530@ringo85308 ай бұрын

  • Wow. Just wow! This is brilliant🤩

    @mariusschmidt6883@mariusschmidt6883 Жыл бұрын

    • Thanks!

      @ritvikmath@ritvikmath Жыл бұрын

  • Thank you very much! Besides the "norminal" category in your example, I am wondering if this can also be used in "ordinal" category. For example, if I make a questionare from "dislike" to "like very much" and get poll from 2 groups, can I use the KL-divergence to calculate the difference between these 2 groups, and whether there is a even better way to discribe this difference, for example, group2 shows "significant" higher interest than group1 ?

    @chenqu773@chenqu773 Жыл бұрын

  • Thank you :) for valuable content

    @DataScienceAI-rf4kx@DataScienceAI-rf4kx3 ай бұрын

  • Great explanation 👏

    @shadabalam2122@shadabalam21222 ай бұрын

    • Glad you think so!

      @ritvikmath@ritvikmath2 ай бұрын

  • great explanation!

    @orenkoriat@orenkoriat Жыл бұрын

    • Glad you think so!

      @ritvikmath@ritvikmath Жыл бұрын

  • Super clear !

    @yorker0507@yorker05073 ай бұрын

  • 讲的真好，谢谢

    @shuweiPeng-id4xv@shuweiPeng-id4xv3 ай бұрын

  • Amazing explanation

    @chandrashekaravula1292@chandrashekaravula1292 Жыл бұрын

    • Glad you liked it

      @ritvikmath@ritvikmath Жыл бұрын

  • I am a masters student in data science and machine learning and I have to tell you that this is the best explanation one can get for concepts like this...Hope you make more videos on these types in concepts.

    @jayalekshmi936@jayalekshmi936 Жыл бұрын

    • Wow, thanks!

      @ritvikmath@ritvikmath Жыл бұрын

  • Great job

    @physicsfaith@physicsfaith10 ай бұрын

  • I think this would be a good prerequisite for Variational Inference and SVI, if you planning to do that vid

    @arvinflores5316@arvinflores5316 Жыл бұрын

  • aaaaaaaaaaaaaa!!!!!, This video is absolutely mind-blowing! thank you for your great contribution.

    @zhibinniu8072@zhibinniu8072Ай бұрын

  • Universities should fire their math professors and get you to teach their classes. Well done!

    @akrylic_@akrylic_ Жыл бұрын

  • That was really awesome

    @RAHUDAS@RAHUDAS Жыл бұрын

  • How can this guy only have 8,000 views on such a good video... Very nice way of explaining!

    @luisgoogle8098@luisgoogle8098 Жыл бұрын

    • Wow, thank you!

      @ritvikmath@ritvikmath Жыл бұрын

  • Great video!

    @jijie133@jijie133 Жыл бұрын

    • Glad you enjoyed it!

      @ritvikmath@ritvikmath Жыл бұрын
KZhead