Convolutions | Why X+Y in probability is a beautiful mess
Adding random variables, with connections to the central limit theorem.
Help fund future projects: / 3blue1brown
An equally valuable form of support is to simply share the videos.
0:00 - Intro quiz
2:24 - Discrete case, diagonal slices
6:49 - Discrete case, flip-and-slide
8:41 - The discrete formula
10:58 - Continuous case, flip-and-slide
15:53 - Example with uniform distributions
18:42 - Central limit theorem
20:50 - Continuous case, diagonal slices
25:26 - Returning to the intro quiz
Thanks to these viewers for their contributions to translations
Hebrew: @DavidBar-On, David Bar-On, Omer Tuchfeld
Spanish: Derek Lacayo
------------------
These animations are largely made using a custom python library, manim. See the FAQ comments here:
www.3blue1brown.com/faq#manim
github.com/3b1b/manim
github.com/ManimCommunity/manim/
You can find code for specific videos and projects here:
github.com/3b1b/videos/
Music by Vincent Rubinetti.
www.vincentrubinetti.com/
Download the music on Bandcamp:
vincerubinetti.bandcamp.com/a...
Stream the music on Spotify:
open.spotify.com/album/1dVyjw...
------------------
3blue1brown is a channel about animating math, in all senses of the word animate. And you know the drill with KZhead, if you want to stay posted on new videos, subscribe: 3b1b.co/subscribe
Various social media stuffs:
Website: www.3blue1brown.com
Twitter: / 3blue1brown
Reddit: / 3blue1brown
Instagram: / 3blue1brown
Patreon: / 3blue1brown
Facebook: / 3blue1brown
Next video: kzhead.info/sun/l8OqprB8n52dY2g/bejne.html
❤
*Me watching this video having no idea what is happening but watches anyways*
Hey Grant. Not to make a request, but I think it's a pretty neat video idea, being a relatively untapped vein of math communication: have you thought about doing a video on stochastic calculus and Itô processes?
now its the first time ive heard about this because i disabled community posts :/
@@Eta_Carinae__ or even more off topic, yet. Your own take on visualizing fractional derivitaves with the Riemann-Liouville, or some other approach? While not apparently useful, a newer math topic like this always is fresh to see a video on. Is extending this idea into the complex domain or R^3 space possible as a visualization?
I wonder how many non-math people never would've thought they'd find themselves on the edge of their seat waiting for the next video in a series on probability theory. Truly a beautiful animation and explanation of this topic!
As someone who hated stochastics in middle school and is now working with applied statistics and machine learning, I just wish these videos had existed sooner 😅 I've always been a fan of geometric intuitions, and this is why this channel does stand out so much to me. Grant has a talent of making abstract things graphical.
@@MattRose30000 seriously though. it all felt like chores when I was a child; the supervisor for the reinforcement learning on us kids could have tuned the model better :P
@@TengzhichongYou made me laugh ... Thanks
Babe wake up, funny math guy just uploaded
Funny?
“I like your funny words, math man”
Cute pie creature !
Yay, new whity math 👀ツ
Babe wake up! Someone just wrote a "Babe wake up!" comment!
Having just come out of a Calculus 1 class, I can look at these videos with a whole new world of understanding. Before, I had watched these videos because I thought it was cool and interesting to know what was possible with mathematics. But now that I have learned how to take a derivative and am integral, I can follow along with the processes much closer, and gain a better understanding of how these tools of calculus are applied to various problems in mathematics. It's much more fun this way, and makes me feel like the effort I put into the course meant something.
Wonderful to hear. Calculus really does unlock a whole new world after you take it, including essentially all of physics
Calc 1 was the first time I was excited to learn math for years. Derivatives and integrals feel less like a mechanical process and more like playing with numbers.
Hell yeah! Isn't that such a wonderful feeling? 🤗
@@3blue1brownor me, calculus clicked in place when learning Physics I - and understanding the relation between velocity and acceleration. How the formulae I learned in High school are *derived* from each other. DERIVED. It was a WHOOOOAAAA moment. The word means more than face vakue. Everything just clicked. Your videos recreate that feeling. And I love it. I do grab pen and paper with your videos and calculate along. Best days!
I think this comment contains a really important point. I often see comments that are like, "wow this explained it so much better than my teacher" "why couldn't you just teach everyone" and things like that, but as flashy as these videos are and as simple as they present the concepts, you can't get full understanding of something in mathematics from just watching it. You have to actually do it, and practice it a lot.
As a civil engineer by trade, the two convolutions I most enjoy are: 1. Convoluting a Unit Hydrograph with a Hyetograph to determine a given natural system's (or, "watershed") surface water conveyance response to a given rainfall event. Then, 2. Using multiple watershed responses (say, individual discharge points from streams), convoluting the intersection of multiple watersheds (streams) to determine a larger river systems response to various rainfall events. The Corps of Engineers has been using the concept of convolutions for decades to create flood probability maps for the entire United States. These maps, which establish the flood level for a given return-period storm, in turn, are used by insurance companies to determine the rate that should be charged for your flood insurance at your particular home. How's THAT for real-world application of convolution?!
I bet wildlife conservation agents use this approach as well for reporting over-population for game based on crash report data. Like how many white tail deer are becoming a nuisance per convolution of crash statistics.
Also a civil engineer! What do you use to make these convolutions?
That sounds pretty convoluted.
As a teacher of actuarial science (insurance mathematics), I cannot wait to share this video with my students next time I teach about convolutions.
@@alejandrotenorio2327 Several different ways, I suppose. The classic approach is that used by the old Fortran-based model, HEC-2 (later, HEC-RAS). However, there are other methods that found popularity after computational power increased. Two are the Runge-Kutta method and Taylor series expansion. These days, one can even apply Monte Carlo techniques to filter out some of the randomness of otherwise stochastic responses in complex hydrologic systems.
*Side note:* I found a really cool method for geometrizing/visualizing geometric integrals. That is taking the function you want to integrate, graphing its square root in polar coordinates, and using the formula for the area inside of a polar graph; this becomes useful if the polar graph draws a conic section, which is actually not that hard to take the area of. *I have r/mathematics posts with examples (listed by title, from least recent to most recent):* • "Yesterday or so, I realized that polar graphs can be used to geometrize integrals..." • "I played around more with that cartesian substitution I discovered a month ago."
That's a really neat way to integrate squares of trig functions, I hadn't seen that before!
@@3blue1brown: The solution for the integral of secant is also cool. It turns into the area of a hyperbola sector.
Isn't this like basically a generalizaion of the famous solution for the Gaussian integral, where you transform it into 2D and then into polar coordinates? That is so nifty!
Nice
Bro I figured it out way before even for discontinuous functions .you take the langharian zeros of the function and put them in the gamma function . Basically this loops the area of function into a circle around origin. From where it's radius can be determined and using pi r square u find the integral. Also my post got 17.9 k upvotes
Another priceless experience paired with a heartbreaking cliff hanging. Thank you for your work!!
I literally started gasping loudly and violently at the cliffhanger. Now can't wait a MINUTE for the next one...
This is the most cliffhanged I've felt from a 3B1B video. He's outdone himself.
Very few KZheadrs make a 30 min-long Math and Science video that is more fun to watch than a 15-second-long Instagram reel. Hats off to all of you!
Having 30 minutes fun is always better than having just 15 seconds :D
I love the moments in his videos where he drops some profound truth (repeated convolution of any function produces a normal distribution), and I can only sitting there grinning in confused wonder at how that could be possible. It's kind of like getting to the end of a novel and reading the "twist ending" and that you never saw coming, but which fits perfectly.
Hey Grant. I know this isn't the right place, but I am really, really waiting for a course on statistics, just like your linear algebra one. The lectures will prove to be gems for me, especially in QM and engineering
Back in the days when mainframes had fairly fast processor-level pseudorandom number generators but relatively slow transcendental functions, a common way of getting a semi-decent Gaussian-distributed variable was just to sum three or four variates from the hardware RNG, suitably shifted and scaled. I've actually seen this in some FORTRAN code for a particle accelerator simulation (which was eventually rewritten in C++ and became PYTHIA).
This is such perfect timing, Grant. I was just studying this from a textbook, and I wasn't able to gain an intuition on continuous convolutions; and here you are, to the rescue! Once again, we cannot thank you enough for your brilliant contribution to the world. Thank You, Grant. ❤
I minored in statistics. I thought I understood everything I needed to know about the Central Limit Theorem. But that visualization with the repeated convolutions approaching a normal curve made it look like such an intuitive, obvious fact. I’d never looked at it that way before, and it was beautiful! Well done!
The diagonal addition representation instantly clicked as convolution, on a part that took me much longer to get when I first learned about conv. All your videos are made of these little moments and insights that are just so spectacular to visualize. Thank you
My man, 3 blue 1 brown loves Fourier transforms so much, that his animation of the eye, his channel logo, is literally converting a function from time domain to frequency domain. What an amazing hidden gem, such a cool way to put Fourier transform animation into you logo. Amazing.
As a person with aphantasia, you'd think I'd be the inverse of the target audience here, but... I find these videos genuinely fascinating. They help me understand how other people conceptualize some of the same things I do, but with imagery instead of deductions from axioms. Great stuff.
Same!
Agreed :) I had a similar thought when my aha! moment for this video was pausing on the Reimann sum text not anything visual, and had a bit of a laugh at myself (then pondered why I like the videos)
I am the 7th-12th grade math teacher in a rural community, and I wanted to tell you that your videos have inspired me to learn Python so I can make interactive educational videos on topics and levels my students can enjoy. Thank you for continuing to deliver great content that inspires a love for math education.
No benifit bro ur rural children won't get any of that stuff just teach em the basics. Why waste money on those bastards only to be dissapinted
Your students are lucky to have you as a teacher!
@@apnatime4831Don’t criticize a good teacher putting forth extra effort. Actually, a teacher is probably what you need, to help you spell better
@@jacksonstenger k DUDE chill 😎 🤙 🤘
Now let's multiply two random variables
I love the topic choice. I love how you're dealing with it. I hate i have to wait weeks for the next episode, but i know it worth it for the quality. I just wish i discovered your channel five years from now, so i had already the full serie. Thanks Grant for what you are doing and providing it here
Yeah, but in 5 years Grant will still be making awesome videos that you'll have to wait for.
I didn't take any math past Trig and these videos make total sense to me. Wish they had this video for me back in 1994!
Perfect timing, just 2 days before my Probability Theory & Statistics final at uni!
I took probability last semester, this would have helped lol. Good luck on the final!
@@WobblesandBean Thank you!
Good luck on the exam; may the nerd force be with you!
Good luck!❤
It's always so sweet to see the intuition you bring to these topics. The smooth way everything clicks together. Probability is integral part of my work (phd in financial econometrics) and when doing advanced stuff it's easy to forget the beauty hidden in the most simple things.
I just finished your Discrete convolutions video and Residuals FFT that you recommended in that video. Was looking for your video on continuous convolutions. This is impeccable timing! Thanks for this!
Man, I love how as I go through high school I understand each new video a little more, it felt like I understood this video fully and was always able to predict what came next. Great work, I really do appreciate you explaining these topics so incredibly well for free.
You have a tremendous ability to hint at what's to come! First identifying the equivalence with the diagonal and then figuring out where it comes from using the formula before you even presented felt incredible, thank you so much Grant for this experience!
It took me 51 years, and a KZhead video from one of the best, but I finally got convolution. And the explanation was not convoluted at all!
Coming from just finishing a Probability Theory course, these videos uncover a whole new world of visual understanding behind the formulas we've been using the whole semester, and its beyond enjoyable to shout "ITS CLT!" after the visualization, and be right :)
I wish these visualisations had been available when I was struggling to get my head round stuff like this 35 years ago! I remember using a convolution integral to solve some Laplace Transform problem in electrical circuit analysis, but being annoyed that I didn't really understand how it worked!
You're making the world a better place, one video at a time. Thank you so much!
Hey, I'm en EE student and just couldn't wrap my head around why a multiplication in the time domain equals a convolution in the frequency domain. With your shown approach of asking the question of what is the area of all the function products of the combination of arguments that equal x and the "sum trig identity" it suddenly is extremely obvious, tysm! ❤
I took my Signals and Systems course for my EE degree a year ago (which was basically just a math course on affine transformations, convolutions, and Fourier transforms on discrete and continuous signals/functions) and this was a nice refresher on the intuition behind convolutions
Every video you release breaks my heart with a cliffhanger 😩 Your content is so good Grant, I never want the lessons to end.
Hey Grant! I just took a course on probability and statistics this semester and this video is a great way to review and reinforce the intuitions I have on the course just before the finals. I would love for you to make a series on calculus of complex numbers, talk about analytic functions, countour integrals and stuff like that. Even though I finished the course on that topic, I would still love for a 3B1B video/series on it and many would be interested too! I also would like to mention that most of the intuitions I have in maths, be it calculus or probability, is because I have watched 3B1B. I have a decently strong idea of what is going on in class because sometimes I can connect what I saw here and what I learnt there. These videos are excellent for communicating maths and my friends and I just love it! Thank you for what you do.
Your videos are weirdly comforting to me. Even if I don't fully get them, I really enjoy watching. Also, you made me really like math, I've been self studying calculus after watching your series on it.
I studied math in university. And probability theory was always my weakest subject. I could never intuitively place the math and its implications in my brain. In almost all other subjects, like calculus, measurement theory, algebra, etc.. I had a clear intuition. Not in probability theory. Its hard to build that intuition. And this series, of convolutions and probability theory is actually plugging the holes that my university education left me with. I would have been a much more successful on the subject when I studied it with your videos to give me a hand. Thank you, Grant. Also, notice how the colors are chosen to be visible for people with red/green viewing disabilities? I dont have that impairment but I notice it nonetheless. Great work!
As someone that looked into convolutions in the past but never quite understood them, this video really solidified my understanding that I couldn't quite explain before. Before I just saw it as a daunting operation that could help me with Laplace Transforms. Now, I can see it more as a 'comparison' operator between two functions. It acts as, essentially, an operator analogous to the dot product for vectors, by comparing how much of both functions at a given point are 'similar', in the same way the direction of two vectors with respect to each other is compared in the dot product. Thinking on it now, I see it almost the same as the idea of the FTC, but the FTC definite integral compares a function to the width of the interval you are integrating on. This acts as a more generalised version of that definite integral (not literally, just for lack of better phrasing) and compares a function to another. Thanks 3B1B, for another cracking video that really makes me enjoy Mathematics more and more by the day.
I have nothing more to say than the pleasant to watch your videos. You make me, a sixth grader understand calculus, topology and a ocean of beautiful math. The world becomes a much better place with your videos sir. Great respect! 🤩🤩🤩
The visuals in these videos deserve to be played on a big screen TV hanging in the Louve. I can't imagine any better use of today's computational power and programming / animation tools than producing these educational videos that not only lift the veil around mathematical mechanics but provide insight into the world around us -- exactly what math is supposed to do.
Your videos are calming and engaging. I never thought that math explainer videos could be calming ...only anxiety provoking or boring or both.
I'm an electrical engineering student and just finished learning FTs for system response stuff and this video has blown my mind to give me a deeper understanding of all the math I did all year. Thank you so much
Finally the probability series we waited for :)
Exactly! and he started it without letting us know
I begrudgingly took 6 months of Control classes for mechanical engineering, which is basically just lots of analog signal processing mathematics, and i don't think any of the subjects stuck. Demented unmotivated teachers didn't help, of course. Your videos have actually sparked an interest in this field for me, and made me understand stuff. Thanks man.
You've really outdone yourself. My signals and systems class years ago would've been so much more... accessible? with these videos as an aid. Glad current students are able to benefit!
I have been exploring math on my own in the past month, and I have realized how many things could be geometrized. (Kind of a side-note)
Your mom could be geometrized
@@idontwantahandlethoughwhat are you 12?
@@idontwantahandlethough new to internet boy ? Huh
I'm 60 and I thought it was funny. Your mom is probably 12. Anyway, Inspirator's comment reminded me of how, to the Ancient Greeks, numbers were geometrical objects.
It’s always great to see how you bring in geometry to generalise and make seemingly abstract concepts become intuitively obvious. Fantastic teaching technique!
I'm genuinely in love with this video. I got obsessed with Monte Carlo simulation a while back and this is amazingly useful!
You always show me new ways of thinking about tools that I have used for years. Thank you.
Thanks a lot for this video ! It might be far-fetched, but I work a lot on audio synthesis these days (programing my own synthesizers) and while I use convolutions A LOT (for effects, mainly), I didn't quite understand how it worked until your video. I'll have to watch it three or four times again, and make more researches, but I feel like something "clicked" while looking at it. Awesome stuff, thanks a lot.
I find your videos absolutely amazing! Thank you for the time and effort. The moving graphics are so well done.
This video is beautifully made. I'm a university student and one of the courses this semester was a statistic course. This video was uploaded a few days before the final exam, a great way to sum up what I've learned in the past 3 months
This is seriously better than a proper university lecture on the topic. Thank you for this video.
Yes! Your visual Linear Algebra series was transformative for me, and I get the feeling that a similar series on mathematical statistics will also be.
Well said.
Oh wow, I'm someone who doesn't usually chime with visual explanations, algebra tend to resonate better with my understanding. But I was fully engrossed in the visual, kind of ignoring the algebra, and I literally said out-loud "that's anti-derivation, it's integration" and then looked to the right of my screen to see an integral. Brilliant work, as always.
Every time I learn about convolution, some amazing new thing surprises me. Thanks a lot.
Love this channel! Epic work on the math and the animations Grant! I'm studying path tracing in my little free time, this is all highly relevant!
I'm graduating with my BSEE degree, and this would have been extremely helpful for a couple of classes. Very insightful for you electricals that haven't done linear systems, or want to focus in communications.
The best part about going to harder and harder math classes is being able to rewatch your videos and know what on earth your going on about
Thank you for this one! Can't wait to see it when i finish working today!
You've got some serious cojones putting this out the day before my probability & statistics exam.
❤ Thanks Grant! Nice to have you back!
This is one of my favourite videos so far! Thank you!
This is amazing! Such an overwhelming amount of profound realisations hit me while watching this. Thank you so much for your videos
Probability is really mind-blowing. There are rough analogues of CLT's that result in a distribution that is not normal i.e., The Tracy-Widom distribution, Wigner's semicircle distribution etc.
Everytime you make may 50 years old engineer mind explode with yourt wonderful videos! Thanks !!
Its probably no surprise to you but i think you should know that the videos you do and have put out throughout the years immensely help those of us who are currently or about to undergo a mathematical heavy education. In my case i am in Area Studies (middle east & north africa) but will be leaning into economics and hence these maths videos are insanely helpful to understand maths and statistics better. your content is super inspirational and im very happy to be here to witness it, thank you so much
I just took more than 1.5 hours to 'somewhat' understand a 27 minute video, and at the end of it, I can say that I understand 1% about convolutions. It has been a while since I have watched a complicated math video and simultaneously understood everything that has been said, but in this case, I did understand almost everything but for 3 things. This video is on the level of being a research paper in itself, it's so well made. The animation, the code that went in, the script and the approach to not bothers the viewers with pesky integrals, are as always, a 3B1B signature at this point. But I really hate cliffhangars, so I am already awaiting the next video in this series.
The convolution has been de-convoluted by this beautiful intuition.
My "aha moment" was when I realized why we see normal distribution so frequently in nature - nature has an insane amount of variables convoluted together 😮
Ah, 14:00 - 16:00 was so good. The explanation of "Where's that y gone?" and the joy in seeing how adding together 2 graphs of fixed shape can result in something resembling a travelling wave(let). Come away feeling inspired!
Really educative way to introduce the convolution. I loved this video, thanks.
Amazing synchronicity - I was just researching how to make non-uniform random number generators in an electronic music application. Did some experiments in Python and came up with a lot of the same graphs as those shown here.
The visuals have reached a new level. Really well done.
I haven't even started watching yet, but dude your awesome. I literally needed to learn the premise of the refined version of this in base 10. thank you!!!!
"an attractive fixed point in the space of all functions" Wooahhhh, that was a great insight/way of framing it
This series has already taken 5 videos, keep it up!
amazing insight, superbly explained with your soothing voice - a great mix of enthusiasm with a calm energy!
I am currently reading 'Statistics for Experimenters' (Box, Hunter, Hunter), and just read the section on this. Your video is a really nice visual and accessible rendition of the content.
Where were these videos when I did my undergrad! I hope this elegancy and beauty inspires more students to continue.
The nice thing about the diagonal representation is that you can rotate the 3D graph so you're looking along the slices instead of through them. Then you can mentally squish the slices and form a 2d graph, sort of like the trash compactor in Start Wars. Each slice gets squished into a vertical line segment.
When I first learned about convolution I was told to "slide one graph along the other" but this trick never made much intuitive sense. Thank you so much for explaining convolution intuitively.
Thank you for the video. I studied convolutions for my degree and passed the course but I didn't have a picture in my mind about what the convolutions were doing. This video has helped a lot.
Great video. Wish your videos existed when I took stochastic processes!
Hey Grant, i really enjoy your videos. Your explanations from simple examples up to the general concepts are interesting and feel natural. The understanding growing in mind is so satisfying. With no destraction by strict mathematical definitions, i find it easy to follow. Also the amazing animations arent just nice to look at, they do a great job in supporting the intuitive understanding. You fill the gap of explanations, that are missing in my university courses. Thank you for your work, im looking forward to the follow-up video ✌
Thanks!
Having studied AI your whole channel sums up my study in an so much easier way. Our teachers over complicated stuff or didn’t even bother to explain the underlying mathematical theories of the machine learning algorithms. So thank you very much sir. I am going to watch every single video☺️
My first intuition on hearing the initial question was to plot it on a 3D graph, then somehow project the volume under the graph into the area under a 2D graph. It's always fun to see an intuition confirmed, and explained in such detail.
Very informative and well described. Thank you very much for this!
The best math channel by far. You rekindled my passion for math, thank you for the amazing content!
4:39 thanks for saying this. I think KZhead should require a warning for all CLT/IID videos that as mathematicalily satisfying as they are; very often the real world isn't like that.
Beautiful and wonderful video! Thank you for the clear explaination!
As a music lover, I applaud the juxtaposition of Vince's "Occlusion" with Rubinetti's "Heartbeat"; and as a math lover, I welcome the 3D visual for how to apply continuous convolutions to different normal distributions. More!
i have to admit that your videos are challenging to watch because i am not good at math, but the reason i watch every video are the beautiful anomations.
Thank you for your visualisations. I can NOT follow the math, but you always make me SEE what you are doing even if I can not get there myself.
Super cool! I never thought like that! You explained to us the simply deep reason why the convolution is used!
You’re my hero. I’m quitting my corporate career to start my own business teaching math and excel. You and StatQuest are my inspiration.
oh nonono I need the answer now! Truly beautiful and insightful, this video kinda revolutionized the normal distribution for me. Thanks!
I've been playing with binary multipliers the last few weeks and it's fascinating to see convolution at the heart of long multiplication. Great video!
I wish I had seen this when learning all this: more fun and easier to remember than calculus class. And it makes it really clear where the square root of 2 comes from.
You are awesome teacher! I have a Ph.D and I still enjoy the content and benefit from it due to deeper understanding.
Beautiful, thank you! So relaxing and good quality content, thank you again! :)
I am currently attending the first year of physics at uni and tomorrow I'll have to do the oral exam for my stats course. The professor explained continuous convolutions on the last lecture and this video just dropped... I think I'll carefully watch the video and be more thankful to Grant than I've ever been.
best of luck for your exam ❤
@@abirsadhu5538 thx