Python 3.12 Generic Types Explained
Check out Hostinger ➡️ www.hostinger.com/arjancodes
✅ Hostinger code for 10% off: ARJANCODES
In this video, I’m going to explore how generic types in Python 3.12 work, and what the advantage is over just using the Any types.
GitHub repo: git.arjan.codes/2023/generic
👷 Join the FREE Code Diagnosis Workshop to help you review code more effectively using my 3-Factor Diagnosis Framework: www.arjancodes.com/diagnosis
💻 ArjanCodes Blog: www.arjancodes.com/blog
✍🏻 Take a quiz on this topic: www.learntail.com/quiz/jqonfv
Try Learntail for FREE ➡️ www.learntail.com/
🎓 Courses:
The Software Designer Mindset: www.arjancodes.com/mindset
The Software Architect Mindset: Pre-register now! www.arjancodes.com/architect
Next Level Python: Become a Python Expert: www.arjancodes.com/next-level...
The 30-Day Design Challenge: www.arjancodes.com/30ddc
🛒 GEAR & RECOMMENDED BOOKS: kit.co/arjancodes.
👍 If you enjoyed this content, give this video a like. If you want to watch more of my upcoming videos, consider subscribing to my channel!
Social channels:
💬 Discord: discord.arjan.codes
🐦Twitter: / arjancodes
🌍LinkedIn: / arjancodes
🕵Facebook: / arjancodes
📱Instagram: / arjancodes
♪ Tiktok: / arjancodes
👀 Code reviewers:
- Yoriz
- Ryan Laursen
- Dale Hagglund
🎥 Video edited by Mark Bacskai: / bacskaimark
🔖 Chapters:
0:00 Intro
0:22 Generic Types
5:41 The Any Type
7:57 Old vs new generic type syntax
12:16 Upper bounds
16:02 Advantages of Using Generic Types over Any
17:58 Outro
#arjancodes #softwaredesign #python
DISCLAIMER - The links in this description might be affiliate links. If you purchase a product or service through one of those links, I may receive a small commission. There is no additional charge to you. Thanks for supporting my channel so I can continue to provide you with free content each week!
Check out Hostinger's Black Friday deals here: www.hostinger.com/arjancodes. And don't forget to use coupon ARJANCODES for an extra 10% off!
'Generic type' was my nickname in highschool
You could then become anyone 😉
😂
Python team following Go typing system and I AM LOVING IT, we can get GO and Python to work the same way LETS GOO
I was googling 3 days to understand how to create properly custom types. How to type hint my function/class for using particular class or its subclasses (ancestors). Because type[Baseclass] is not what i really need. And thanks to the video, now i understand that i have to create my custom type with TypeVar and bounding arg to limit the ancestors. Thanks !
I'm glad the video was helpful, Ivan!
Thank you for the excellent presentation and professionalism. You are the best IT blogger on KZhead!
I appreciate the sentiment! Thank you so much.
Love this! Arjen how would you define a numpy array of objects (class instances)? would you import from numpy.typing?
Can you make a continuation of this video about TypeVarTuples and paramspec? I had a little more trouble understanding these.
Thanks for the update, this change is welcome. Also, Python starts to look more and more like C++ 😀One note: in the video you often say "you can" or "you can't", or something along those lines. If I'm not mistaken, you always "can", as annotations are still just for the IDE + SCA not for runtime. But that is maybe not as clear for people learning the language. (And I assume your channel rightfully attracts many beginners) So maybe clarifying that in each typing related video independently from precious ones could be beneficial. Just a humble suggestion, and thank you for the update again.
And C++ looks more and more like Python 😊 I see it as a win-win
@@yyyy-uv3po do we converge to THE programming language?! 😀 Just kidding, Uncle Bob presentation came to my mind. Regardless. Where do you see C++ becoming more and more like Python? (Honest curiosity, not mocking)
@@MateHegyhati The for() loop, the ability to return several parameters and unpack them like in Python, span gives a bytes vibe, optional gives a None-like value, etc In general, I find that my C++ code reads more and more like plain English, and they managed to do that with very little overhead IMO.
@@yyyy-uv3po Hm... I never felt those to be "Pythonish ", maybe because I learned them before familiarizing with Python. For me "and/or" is what comes to mind in that regard.
great overview, thank you! I love the changes.
Thank you for the kind words! :)
This is great. If possible, I would like to see more advanced concepts introduced in videos like this, eg, connect it to type theory if possible :)
you've finally clearly and easily explained to me what generic is, God bless you
Could you explain the After.py?
Glad you enjoyed the content!
@ArjanCodes Could you tell which plugin do you use for AI assistance, please?
This is a very clear and useful introduction to Python 3.12 typing. It has primed me to dive into the docs. Thank you!
Glad the content motivated you!
What code completion tool is that ?
I used hostinger a long long time ago... its good experience
Very good content - thank you!
Glad you enjoyed it!
Fun to see how similar scala 3 and python syntax are getting. 😂 Thnxs for the clear explanation
Thanks, really useful information
You’re welcome!
Thank you
Stupid questions here : I am using vscode, how the does your ide suggests complete code like at 0:57 ? Also, I am still wondering about the need for type hints, mainly because my vscode does not use them and does not know of which type a variable is, how do you make it actually read the type hints ?
Hey which videos editors you are using. And can you make video on how to making reels thanks!!
His info is in the description of the video!
firstly , great video as always, super cool and good explanation . Secondly, could we have another video about upper bound type ? That example at 12:30 ,, that is so java thing (nominal type , asking who is it before what it can do) , could we make another example with duck-typing with protocol and upper bound ? that would be great .
wait... how does code complete in 2:00 is working? Is it base on the docstring?
Is this autocompletion of the code done by CoPilot or is it another auto complete extension?
The real upshot of generics in python: if you are stuck with using python, then they are useful. But if you can choose any other programming language, then choose a properly statically typed programming language, like Haskell say.
So generic classes are covariant by default? Can you define invariance/contravariance with the new syntax?
Does anyone know what is the shortcut to trigger the type import hint at 3:01 ? Thanks in advance! 🙏 Edit: Closest I can get is `Ctrl/Cmd + .`, but that still looks different 😅
Good explanation, Liked the new feature in python. Kudos 🎉 code link?
im so into typing hints on python
Great video, thanks! It looks like inheritance in this case means "uses an X" instead of "is an X"? Can't wait to use the new syntax once all my dependencies catch up to 3.12.
Thank you! Conceptually, generics are not inheritance. This is also why the pre-3.12 syntax of needing to inherit from a Generic class is confusing. Generic means that the class (or function) is built around one or more types that you only specify when you actually use the class or function.
Thank you Arjan, I wanted to spend my Saturday with clean the house and do some sports. Now I'll code instead. You always make me willing to work on my pet projekt :D
You're welcome / I'm sorry? One of the two ahah! Thank you for the support :)
1:58 is that pair autoprogramming?
nice video!
Thank you! Cheers!
I wonder if there are there ways to reflect the generic type T to do runtime validation. Eg ‘is_inst(x, T)’ where is_inst is a user defined function that returns a bool.?
You can do that by using a @runtime_checkable Protocol
Thank you Arjan. I've cleared in my head now how to use TypeVar.
I'm glad the video helped!
@@ArjanCodes Hi Arjan, I'm a follower of your channel and I eagerly wait for your new video every friday and you've taught me the right way to write python code which saves me a lot of time go thru' the books. Thanks. Adam
I started using type annotations less than a year ago (despite using python since the last millennium), and I LOVE IT. To be honest, one of the reasons I love it is that type annotations do not force me to behave, but it makes it so useful to behave. And generics are phantastics: It is simply fun to carry a concept to the meta-level. For an educational plotting tool which I've written in the last days (called schulplots), I first forgot to turn on type checks in my IDE. Bad idea. I had so many stupid errors -- I've almost forgotten how cumbersome it is to write non-trivial code without type checking turned on. Just today, I stumbled into a situation where I was almost forced to turn off type checks in my IDE: I derived a class from a dataclass, and pylance did not understand that the derived class had all the stuff which the parent also had. But there came the @dataclass_transform decorator (which is available in Python 3.11, but you can quite easily mock it for older versions on import errors). And voila: Nothing red in my IDE.
Although, if you turn on type checking and are using a package like pandas, your whole code base explodes with red squiggles even though everything is totally fine. I usually leave it off because of that, unfortunately.
honestly I couldn't disgree more. A developer that needs type hints to understand it's own code, doesn't know what he/she is coding. And a developer that in joining the team will soon enough become 'the developer that is developing that code'. So the window of opportunity for type-hints to be useful is rather limited, and applies only to new-comers of a team with a relatively large code-base to maintain, and that's only for a couple of months, until that new person has everything fallen onto place.
@@liquiddddd oh yeah, my memory exists with only one purpose to remember every function I've written. 😂
@@avimir8805 oh yeah, and type hints DEFINITELY allows one to have no memory whatsoever of it’s own code. 🤣
What are you using to generate the code from comments etc?
GitHub Copilot.
Nice video as always @arjan. Since you‘re a type junkie - how do you think about mypy? Maybe it‘s worth making a video about it? :)
It's nice that in Python types are not a religious beings but just another form of communication. I give as an example usage of types in Pydantic and FastAPI where using them, is just a natural way of solution creation process and in return you will have a free validation, value coercion and many more niceties. I do agree with you, that in Python types are opportunistic. Opportunistic is good in programming.
Please, be a type junkie! (and doc junkie, too) Good explanation, that's the reason I come back regularly. Although, this time, it was tooo muuuch aaad.
Watching this I bounce back and forth between "I am a bad dev because I don't write such clean code" and "All this to make some squiggly lines go away..."
Wouldn't Any not be a more usefull anonation for something like elements at even/odd indicies? Because for me list[T] would indicated, it does not matter what the type is but it must be consistent, but what if it isn't, f.e. you have ints, floats and strings. I know Python allows piping up list[Any] seems much cleaner then list[int|float|str|...]?
Interesting that they didn't use [V: Car | Boat] syntax, since it's more similar to the way you specify multiple types that are valid for function parameters.
[V: Car | Boat] will allow values like [car1, car2, boat1, car3] [V: (Car, Boat)] will only allow a list with elements of the same type
@@matchashining7300 ah, thanks for the clarification. I was thinking the generic would be allowed to be only an object of car or boat, not also a list of car or boat.
I wonder what exactly is happening underneath to accommodate generic types. If int and strings are different sizes in memory, then what is the interpreter doing to the memory to make that work.
If I understand correctly, typing in python is only for the IDE and does not impact the execution of the code. Python will bundle most "objects" into a pointer.
Please, can you cover descriptors?
That generic type upper bound pattern is so cumbersome. Protocols/interfaces/traits all do that in a way cleaner way.
Box[T] is one of the ways to solve type variance issues when working with generics, when T is used as both input and output of a generic function. So I would not call the Box[T] useless, more like "niche".
"Because a coffee machine is not a vehicle." Could you please try with Toaster class, whose instances often fly? "Do you use generic types in your code?" If we are making a tool, like a library or a framework, for making software, it is worth spending time to declare precise type specifications using generics. As clearly explained in the video, generic type is a great tool for reusability. In other words, we have less chance to use generic types when reusability is not a goal.
Now, update the Monadic example from last week using the new generics 😝
Hey, What ide you are using?
It is VS Code. Im interested in which extension he is using for code autocompletion and generating functions out of comments
@@alvaroe2704pretty sure it's the git copilot extension
I believe it's GitHub Copilot
@alvore2704 i think it is github copilot
@@alvaroe2704 the ms python extensions for core python support, copilot for the big code suggestions
Arjan: You don't want to put a huge complex type into a single type alias. Numpy: 👀
no doubt, your Python vids rises up YT's average level ;-)
Thank you for the compliment!
The trap I fall into with generic functions is when two object types need to be processed SLIGHTLY differently. That results in small, hard-to-find sections of code within the generic function having to perform runtime type checking of "if type is A then do X, if type is B then do Y". At that point I start wondering if two separate functions would be more clear, even if the code of them is mostly identical.
If you can isolate the differences into methods on the types, then you can use the upper bounds shown in the video to ensure you have those methods on objects passed to your function. Or you could turn the function into a method on the superclass for a fully OO approach. Or two separate functions, as you said. Sometimes code doesn't need to be generic.
Thank you again for another great video! The only thing I do a bit differently is I divide my implementation into .py files and the annotations and the documentation into .pyi files. The reason is that I have colleagues with different python versions on their machines, and if they execute code with new styles they get SynatxErrors all over the place. Since I dont want to force anyone to switch to anything I found this approach to work, Intelisense helps them even if they use versions like 3.10, 3.11 And the code works :)
Having different team members use different Python versions is a recipe for disaster. The project should define the Python version it targets, and everyone should use it, possibly with tools like pyenv.
I feel like Pycharm does most of this automatically, without having to use typehints. Why are you using vscode with pylance if it is buggy?
14:46 wait, you're saying we don't need a coffee machine?
@ArjanCodes I love your videos, but I think it would be better to turn off the AI code completion. I find it quite distracting seeing all the corrections. You are fast enough at typing what you want😊
The generics now like that of typescript.
Niiice
Is that a Durgod Taurus?
Thank you for video. Just thinking aloud: python has gained its popularity as an easy to pickup language with low language-related cognitive load. E.g. there is even no need to declare a variable (like one has to in JavaScript). But I got a feeling, that if one has to use all these type annotations correctly, then this requires much more cognitive efforts than would be required on the language which is initially statically typed (like Java or C#).
In Python, it's true you can cruise without declaring variables or using type hints, making it feel like a breezy ride. But remember, while you're enjoying the scenery in your 'non-production' car, the absence of type hints could set you up for a bumpy road ahead. On D-day, when your code hits production, that's when you might find yourself wishing you had used those type hints. It's all fun and games until untyped code causes unexpected crashes in production!
Which makes it perfect for both throwaway scripts and large production-ready programs. The typing is optional so you can skip a lot of "boilerplate" if you just need a quick script, but if you're working on a much larger project, you can start to use typing so the IDE and interpreter can help point out issues early on
@@dirtdart81 the type checker, not the interpreter
@dirtdart81 I understand that. But my point is that cognitive load of doing this correctly seems to be higher than using a statically typed system to start with.
Thank you, but could've gained a better understanding of generic typing in 3.12, namely after.py. Please consider that us, average coders, could use more low level breakdowns to be able to follow.
You can achieve the same by using list instead of list[T]
No, you can’t. By using list without the generic type you lose the type relationship. So for a function that gets a list and returns a list, there’s no guarantee that those lists have the same type of elements. The function could accept a list of integers and return a list of strings for all we know. With generic types, you avoid that problem.
@@ArjanCodes This is totally correct if you care about type of elements. In your example you just did list manipulation ignoring type of elements.
The point is that the distinction is important. Process_elements should return a list containing the same types of elements as the input list. Without using generic types, you can’t express that.
These are long overdue.
10:29 ahw, PYTHOOON >3.10 the _perfect_ hodge podge of all kinds of syntaxes
Will python become a static typed language soon?
Never according to the BDFL
17:14 ..yes JAVA
I personally use typing but only to a point. At some point the cost benefit of implementing and supporting the complexity of types outweighs the benefit , especially in Python.
python3 is looking more alike C/CPP.!
This is the problem with Python. Instead of a single clean language definition we have an unending list of variations.
I agree, but I think this is true of many languages now: constant iteration. At least Python3 seems to be managing it quite well.
It’s a good thing that Python changes IMO. Any language that doesn’t continuously adapt to the needs of developers and the industry will be replaced by one that does.
@@ArjanCodes Hi Arjan. It gets replaced frequently, by a new language with the same name. The result is people have to know umpteen dialects or versions of the Python family. I find it grimly amusing that the initial appeal of this language was it's flexible typing, and we see more and more proper software engineering features being added to make Python programs more robust. It won't be long before it has preconditions, postconditions, and invariants, and it's called Eiffel! :-) Regards, Andy
dynamic typing was a mistake imo, it causes more issues than it solves, i prefer enforced static typing, so going from that generics is atleast a step up for me
A…. Ah…Why worry about type at all? Just process the input as a list? Unless using cython, typing is not a good thing.. and by making the type generic you surrender the whole premise
Python is becoming more and more like Java😂
Usually, video sponsor and video content are more linked together :p
Forcing me to watch ads is a turnff
could you make a video about mojo language
I think, we make a mistake with type hinting. What a function can process depends on the methods used in the function and if the variables can handle that. In case they can, it isn't a valid variable. Just my 2 cents.
Can't
@@troncooo409 You can edit your original comment.
Type hinting doesn't cause this problem. If you remove type hinting, your function still doesn't work with the exact same argument types
I mean method hinting instead of type hinting.
Well well, python 3.99999 will probably finally have some "final" typing system 😂
13:26 that seems like a wildly inappropriate place to use a Generic, the whole point is that you can make an abstract implementation that can take in a generic variation of types. If you are gonna make some thing as specific as a VehicleRegistry and you have a Vehicle class, it would be much clearner to merely say it inherits from an abstract implementation of a vehicle.
Why not just define it as process_elements(elements: list) -> list: ??
Because then you don’t specify that the input list has the same type of elements as the output list. For all we know, the function as you defined it may take a list of integers and then return a list of strings.
So why use annotations when other languages are already typed and in the case of Rust offering even generics for enums? The idea of of a easy to manage scripted language is going lost.
because people insist on writing giant complex applications in a "scripting" language and then can't figure out what the heck is happening in the program
@@quillaja Then it would be time to create a new language with traits from Python and a typed language. I love Python, made large projects with Python but at a certain point other languages are better.
@@RalphKoettlitz I agree. But also sometimes you don't get to choose the language you must use. I'll gladly take type hints over no type hints since I'm forced to use Python these days in the GIS field.
I don't understand people who want put type in python code.
Hi! Thanks for the video, a good one again! My opinion is that it is a Microsoft influence. It tries to turn python to a full-blown general purpose language while ignoring the original users of the language: the scientific community. My opinion is that this path will alienate the science world from python. Async-await, this generic implementation, etc are CSharpic, not pythonic. It is not by mistake that it was not added till 3.5 or 3.8 (I cannot remember clearly when it was introduced first, but compared to when it was added to c# shows the difference in thinking between scientific community and developer community). I feel these changes goes against the scientific community (not on purpose, more fueled by arrogance), and opens up new possibilities for competitors like Julia, or maybe a new scientific language will be born. It is not by mistake that researchers don't use c# with calling c codes by marshaling but use python instead. So, this is my opinion and my "prophecy", I am looking forward how it works out in the real life 🙂
Why is adding new helpful features a betrayal of the scientific community? They are still able to write the same messy code with no typing that they always were, while those who want the benefits of type checking are now able to utilize it.
@@katycat5e Hi, thanks for your question! compare the history of c#, and python. When c# 1 came out they apologised and told that generics comes soon. Python came earlier, and they do not cared too much about generics since. Scientific community always avoided those problems that developer's communities solve. Look at the logic how plotly works. it is alien to "normal" programmers. We can do better, look at just the logic behind bokeh. Still, plotly is thriving in scientific communities. Why? My opinion is that because mathematicians, etc solves problems differently then normal developers. So, for them this is a feature that they never asked for, but will alter their way of work. This is why I think it is a betrayal. So, my opinion is that this feature is not intended to the scientific community but for the common programmers. New features will bring new theories and ways of work, documents, books, etc that in this case is foreign for the original community of python users. So, currently it looks like scientific community created their language to their messy codes, but general developers started to like it. Now they came, and they turns the language to a kind they run away from in the beginning when they choose to join the python community. Again, this is my personal opinion, I am curious how it plays around in life.
Shame I can't use this in any library code for at least a year or two.
In your explanation of the reason to use a list of generics, you miss the actual reason. If you just want to pass a `list` of any type, you could just do old-school Python and pass an unqualified `list`. However, that would mean that, for any following code that uses something from the resulting list, the editor may not be able to infer what the type of the elements from the result is. *That's* why you want to type the contents of the list, and that's why passing a generic makes sense, since it makes explicit that the elements in the resulting list are indeed the same type as the elements in the argument.
Omg the sound of your keyboard I hate it
first of all tnx. for your effort ! Why use such complicated structures ? with this generic etc ? i do understand and i do use the small type annotation str int etc. but this ?! why ?! this is python !!! if you like to use strong leng use kotlin or ++ what ever ? but this is python is dynamic leng , this is point of this leng.
downvoted due to ho$tinger recommendation
Why in the name of God, does a class named 'MyClass' need a Generic type? Objects of that class would be of type 'MyClass'. End of story. What am I missing here?
If you ever feel the urge to use type hinting in python you're using the language wrong
Completely disagree, the improvement to readability and the static checks by your IDE make them 1000% worth it.
If you ever feel the urge to write software beyond trivial scripts without all the assistance you can get from language and IDE, then you are using computers, your brain, and your future colleagues wrong.
All I heard is click clack clack, bash bash, clack clack. Mech keyboard are dumb