Autoencoder In PyTorch - Theory & Implementation
In this Deep Learning Tutorial we learn how Autoencoders work and how we can implement them in PyTorch.
Get my Free NumPy Handbook:
www.python-engineer.com/numpy...
✅ Write cleaner code with Sourcery, instant refactoring suggestions in VS Code & PyCharm: sourcery.ai/?... *
⭐ Join Our Discord : / discord
📓 ML Notebooks available on Patreon:
/ patrickloeber
If you enjoyed this video, please subscribe to the channel:
▶️ : / @patloeber
Resources:
www.cs.toronto.edu/~lczhang/3...
Code: github.com/patrickloeber/pyto...
More PyTorch Tutorials:
Complete Beginner Course: • Deep Learning With PyT...
Dataloader: PXOzkkB5eH0
Transforms: • PyTorch Tutorial 10 - ...
Model Class: • PyTorch Tutorial 06 - ...
CNN: • PyTorch Tutorial 14 - ...
~~~~~~~~~~~~~~~ CONNECT ~~~~~~~~~~~~~~~
🖥️ Website: www.python-engineer.com
🐦 Twitter - / patloeber
✉️ Newsletter - www.python-engineer.com/newsl...
📸 Instagram - / patloeber
🦾 Discord: / discord
▶️ Subscribe: / @patloeber
~~~~~~~~~~~~~~ SUPPORT ME ~~~~~~~~~~~~~~
🅿 Patreon - / patrickloeber
#Python PyTorch
Timeline:
00:00 - Theory
02:58 - Data Loading
05:30 - Simple Autoencoder
15:02 - Training Loop
17:00 - Plot Images
19:00 - CNN Autoencoder
29:12 - Exercise For You
----------------------------------------------------------------------------------------------------------
* This is an affiliate link. By clicking on it you will not have any additional costs, instead you will support me and my project. Thank you so much for the support! 🙏
Let me know if you enjoyed the new animations in the beginning and want to see this more in the future :)
Hi Patrick, Any plans to have a series on NLP state of the art models?
They're great
Love the animations and want more. Learning pytorch and following your tutorials.
big thanks to you. i cannot imagine how could i learn my dl course without your tutorial. Your work is the best in youtube so far!
I love all your PyTorch videos, please do more :D
Thanks! I will :)
Nice, was waiting for this :)
Hope you enjoyed it!
Great work!
Thank you so much for clear presentation of Autoencoder!
glad you like it!
Great video! Could you provide the same walkthrough for a variational autoencoder? Or point point me to a good walkthrough on the theory and implementation of a variational autoencoder?
Its reaallly nice but it would be a very nice addition to include variational autoencoders and Generative adversial networks as well :). Maybe they can be helpful to many struggling with class imbalance during classification
Great suggestion!
Thank you for your nice tutorials please do the same for a non-image data. I'm curious to see CNN auto-encoders with non-image data.
Hey Patrick a really informative and concise video! Thoroughly enjoyed it :DD Just a small correction at 12:51, you used the word dimension while explaining the Normalize transform, whereas the two attributes are just the mean and standard deviation of the resultant normalized data.
thanks for the hint!
Very clear, thank you very much
hello, nice videos you have. looking forward new videos on paper review and implementations.
at 22:17 when calculating the shape of the conv output, it should be 128*128*1 => 64*64 * 16 and the rest should also be different accordingly
Great work!! Thanks :))
TopNotch explanation! Thx.
Very cool. Thank you.
Nice explanation
Yet another Pytorch video🔥
Can u help to implement pytorch with django
man you are fast :D
Can u upload Ur GUI chat bot code in GitHub I tried code along approach I was able to learn the concept but I got some few bugs . Can u upload it
I added the code here: github.com/python-engineer/python-fun
@@patloeber thanks
Great animations my suggestion is to add in more animations not only in theory but also in the working of the code . Just my suggestion but great video thanks for Ur teaching.
thanks! definitely a good idea
Thank you so much , you explained it really good.
This channel is really good, I learned PyTorch from this channel. Guys I assure you subscribe to this channel.
Thanks so much:) appreciate the nice words
Hey Patrick I used your exact code to train the CNN based autoencoder but couldn't get it to converge without Batch Normalization, after adding BatchNorm2d after every ReLU it works fine, but without it, it doesn't, tried different values for lr from 1e-2 to 1e-5. I was training on MNIST dataset only. the loss becomes NaN or ranges between 0.10 to 0.09.
is it possible to use sentences as input data?
you used recons and img as input for loss function, however if we want to train my model and test it we should use "recon" and "labels" as an input for loss function. but the labels are 3D, how we can do that?
But how do we leverage the low dimensional embedding given that they represent the PCA?
which one is the link that explains how you make the pytorch classes please?
Can u make a video on DNCNN
It should be noted that the performance difference between Linear and CNN as shown here comes from the chosen compression factor. Linear chose 12 Byte per image, CNN chose 256 Byte per image, where an original image is 784 Byte. So, the CNN code does not compress enough, less than PNG actually! You need two more linear layers to compress 64 down to 16 and then 4.
Bro always waiting for your pyt🔥rch video ....🤙🏼🤙🏼🤙🏼
🙌
your tutorials are amazing! Thank you so much... Could you please make a video for nn.embedding as well?
I'll have a look at it
Thanks ❤
hi, I have a question: if we pass the image as input of the model, it will put the weights to zero and then the output will be exactly the same of the input image. So, why the image is given as input of the model? It doesn't make sense to me. Could yu explain this to me?
How you found that the image size was decreased from 28 to 14?
I don't understand a little bit the sintaxis. Why do you define the method 'forward' but never call it explicitly ? Maybe the line "recon = model(img)" is where you are using it, but I didn't know that it could be done like this. I would had written "recon = model.forward(img)", is it the same ?
Beautifully explained!! thank you!
thank you!
You're welcome!
If you normalize the input image which is also the label, the values will be between -1 to +1 but your output since passed through sigmoid will be between 0 and 1. How will you decrease loss for pixels that are between -1 to 0 as your predictions will never be less than 0?
you need to change sigmoid function at the end to tanh, to output will also be betweet -1 to 1
This channel is so underrated.Please upload tutorials about Django
For python 3.11+, pytorch 2.3+ change the dataiter.next() to next(dataiter)
Can you please show how to work with variational autoencoders and applications such as Image segmentation.
will look into this!
@@patloeber Thank you. Because for me it's hard to program it.
why there is no require_grad there?
youre the best
thanks!
Could you please make GANs, VAE , transformers and BERT in pytorch
I tried to rerun your code in the part of Autoencoder CNN, but then I got Loss = nan in each epoch. Do you know what is wrong?
Do you have anything specific for face images, this won't work on face images.
Very nice but need more
module 'torch.nn' has no attribute 'ReLu' Is anyone else getting the same error
its ReLU
@@theupsider Thanks a lot!
Next video is on GAN
Am I the only one encountering nan values during training this ?
On one of my virtual machines i also get nan values, when using torch 2.0.1. I tried a couple of stuff, but only with torch 1.12.1 it works fine. On my desktop pc it works with torch 2.0.1 though. But i do not know why.
dude my CNN autoencoder is doing worse than the linear autoencoder, lmao
me too the ouputs show strange padding artefacts as well
@@marc2911 On one of my virtual machines i also get nan values or strange artifacts, when using torch 2.0.1. I tried a couple of stuff, but only with torch 1.12.1 it works fine. On my desktop pc it works with torch 2.0.1 though. But i do not know why.
"Jew-Pie-Tar notebook"