Protocols vs ABCs in Python - When to Use Which One?
In this video, I’m revisiting Protocols and ABCs in Python, essential for creating abstraction layers by defining interfaces. I covered this a while back, but it deserves a fresh look to clarify: what are the key differences, and when should you use each?
🔥 GitHub Repository: git.arjan.codes/2024/protocol
💡 Get my FREE 7-step guide to help you consistently design great software: arjancodes.com/designguide.
💻 ArjanCodes Blog: www.arjancodes.com/blog
✍🏻 Take a quiz on this topic: www.learntail.com/quiz/qqljls
Try Learntail for FREE ➡️ www.learntail.com/
🎓 Courses:
The Software Designer Mindset: www.arjancodes.com/mindset
The Software Architect Mindset: Pre-register now! www.arjancodes.com/architect
Next Level Python: Become a Python Expert: www.arjancodes.com/next-level...
The 30-Day Design Challenge: www.arjancodes.com/30ddc
🛒 GEAR & RECOMMENDED BOOKS: kit.co/arjancodes.
👍 If you enjoyed this content, give this video a like. If you want to watch more of my upcoming videos, consider subscribing to my channel!
Social channels:
💬 Discord: discord.arjan.codes
🐦Twitter: / arjancodes
🌍LinkedIn: / arjancodes
🕵Facebook: / arjancodes
📱Instagram: / arjancodes
♪ Tiktok: / arjancodes
👀 Code reviewers:
- Yoriz
- Ryan Laursen
- Dale Hagglund
- Kit Hygh
- Alexander Milden
- Bean
🎥 Video edited by Mark Bacskai: / bacskaimark
🔖 Chapters:
0:00 Intro
1:03 Short overview
1:53 Abstract Base Classes
5:39 Protocols
9:21 Making Protocols Behave Like ABCs
12:45 Conclusion
14:54 Outro
#arjancodes #softwaredesign #python
DISCLAIMER - The links in this description might be affiliate links. If you purchase a product or service through one of those links, I may receive a small commission. There is no additional charge to you. Thanks for supporting my channel so I can continue to provide you with free content each week!
💡 Get my FREE 7-step guide to help you consistently design great software: arjancodes.com/designguide.
I generally prefer Protocols to ABCs. With an ABC I always feel like I'm telling the user "You must use my class as a Base class, or else!" A Protocol just specifies the interface requirements without getting pushy about the implementation. As far as concrete functions in an ABC, you can always code them as external functions taking the Protocol class as an argument (like "object oriented programming" in C or Go). Instance checking doesn't resonate with me, because if your code is overly dependent on that, maybe your design isn't so great.
I absolutely agree with you and I couldn't have summed it up any better. To me, protocols are my go to choice when designing interfaces as they just seem more pythonic. Additionally, I absolutely dislike the high coupling ABCs introduce.
For how I understand it: ABC is just a groupment of whatever some classes have in commum in a separated class that holds those behaviours/methods and caracteristics/attributes. Interface / protocol is a special abstract class that forces the subclasses to implement the behavior (implement the interface contract). We only talk about behavior (methods) here, we use it when we want to give some behavior to different objects withtou thinking about their types. It makes polymorphism quite easy as the subclasses can have different implementations of the methods.
My practical conclusion to this question is basically, if the classes you want to interface with follow a strict hierarchy and are implemented with your specific application in mind, use ABC's. If you want to interface with more generalized classes that may or may not be intended specifically with your application in mind, but you do care about them implementing some specific methods, use protocols. TLDR: Protocols if you want to interface with other peoples classes ABC's to interface with your own classes
Yes, I agree a 100% That's exactly what protocol are meant to do. It is not about if you like inheritance or duck typing. It is about having control over the source code or you don't If you do: ABC If you dont: Protocol Thank you. (In Michael Scott voice)
I would advise to almost always use protocols (or at least use them by default) because they are basically abcs, can optin for the few missing features (like isinstance) but like an other comment stated, they are fat better at checking errors at compile time instead of runtime
@@kaosce i didn't find that comment What are the benefits of protocol over ABC?
@@kaosce Well I do agree that there rarely is a good reason to require a strict hierarchy to justify the extra headache. So yes, use protocols in the default case. If you do tho, it would be really dumb imo to have the ambiguity of wether something is actually following the hierarchy or just has the same method names
Protocols also if you want to avoid multiple inheritance. You must avoid it.
I tend to use Protocol where implicit typing is most helpful; e.g., plugins or adapters. And don't forget the 3rd option - use neither. Polymorphism can be implemented without ABC. Just keep the differences in mind. E.g., without ABC you lose the @abstractmethod, et al mechanisms. In general, we have learned over the years that deep type hierarchies lead to code that is less maintainable. Instead, we should prefer shallow hierarchies and only where needed - delegation is more preferable than inheritance. Another great video!
As said in the documentation runtime_checkable is only used for isinstance and issubclass, and only check for attributes and methods, not the arguments of the methods, and even less their types. A type checker, like mypy, is needed for that. Python doesn't care about type annotations and Protocol is meant to be used with a type checker
Yes, I think a good question to ask oneself when deciding which one to use is "are users of your function/protocol reliably going to use MyPy"?. If not, you might be creating a footgun.
With the caveat that runtime_checkable on a Protocol with attributes does not work with issubclass but isinstance
I recently refactored a large plugin with a lot of Protocols . I gave up with the "Writable" explicit naming type and put a 'I' in from of all protocols, like it is done in java, C, etc ... e.g. IWritable . I found it cleaner to read and type at the end. Cheers
I have been taking your videos for granted, but not anymore. You're doing some awesome work here, man!
I appreciate that!
As always, super useful video!
Glad you liked the video!
Could you make a video or short on using callback protocols?
Very informative, please create more system design videos with hands on implementation
Can you do a video about isinstance()? Why it’s bad, how to avoid and situations where it is maybe rational?
It has to be used only when necessary because in python a lot of object are "pythonic" which means they behave in the same way despite being completely different. (think of Tuple and list for example). If you create an object and make it's behavior pythonic enough but you check "isinstance" at somepoint, you will prevent that code to work with other object that would be alike yours.
I love this kind of videos. It's becoming increasingly clear to me that object-oriented programming or "traditional" object-oriented programming is a bit obsolete. Adopting a more TS-functional approach adapting SOLID principles and duck typing is the way forward at least you aren't creating a new framework or something like that.
LOL.. Love your videos.. here in the US, the saying about ducks is "If it walks like a duck and talks like a duck, then it's a duck". Your variation of 'tastes like a duck' make me chuckle.
Arjan's variation of the saying is meant as a joke 🤣
Yep, so Donald Trump is a duck then. It's established.
In the UK it's walks like a duck, quacks like a duck 🦆
Several months ago I needed to pass not instance of implementation of abstract class but rather a class itself. Can't remember why, if I remember correctly, I needed to create instance inside function that has parameters for that class or something like that. To do so, in function signature I needed to type hint it as type[AbstractClass] therefore I could pass any kind of classes that implements all AbstractClass abstract methods. But I couldn't. That's where Protocols can be really handy😅. No problems at all. Still, may be incorrect in details, it was long time ago and I'm writing it using phone in my bed, so I can't really check what the problem was, but I'm pretty sure I described the problem thoroughly. I guess, this difference should be in the video too. Nowadays I don't really use abc... Don't like inheritance + in my code there is really different objects that needs their own implementation for nearly everything, so I kinda lucky that abc is not really better for me than Protocols I like more.
Pyright is checking the output type for protocol's method unless method implicitly write '...' for content. MyPy doesn't. I think it is safer to add the '...' in all Protocol methods.
Protocols are similar to Go interfaces where they are implicitly implemented. This is cool because you are lead to create abstractions when you need it and not when you foresee needing it. The end users get to decide what level of abstraction they want and not be forced into an abstraction level by whoever/wherever else the abstraction was created. "Don't design with interfaces, discover them" - Rob Pike (one of creators of Go), abstractions should be discovered, not created. This wouldn't be possible in many other languages cause of circular dependency etc.
They're both the same concept - structural typing.
Nice to see that python is slowly catching up adding features that prove how it's so much better to have compile-time check rather than runtime errors. You know like any typed languages has been offering for decades. I know this might sound controversial but it is really isn't. The other way to describe "duck-typing" is the John Travolta pulp fiction confused meme saying "you come to me at runtime, to tell me you have compilation errors...". The way u save time is not dynamic typing but type inference.
Hey, one of the cases where we can use the protocol is to describe a module: >>> from enum import Enum, auto from typing import Protocol from my_app.api.adaptors import grpc_adaptor, http_adaptor from my_app.model.user import User class APIType(Enum): HTTP = auto() GRPC = auto() class ModuleAdaptor(Protocol): @staticmethod def get_user(id: int) -> User: ... adaptors: dict[APIType, ModuleAdaptor] = { APIType.HTTP: http_adaptor, APIType.GRPC: grpc_adaptor, } def get_user(api: APIType, id: int) -> User: return adaptors[api].get_user(id) >>> I don't know why, but if you want to write in a functional style, I think you can find a use for it. typing.runtime_checkable does not work in this case, but we will get a hint in the IDE if the module does not comply with the protocol
Yesterday I watched your video about ABC and quickly learned what it is and how to use it. Now I watched this video about ABC and Protocols and forgot even what I learned yesterday 😂
Can you make a video on creating ABC classes with typevars?
I am on the same page - I favour protocol over ABCs when it comes to abstraction, just a few weeks ago however I came to same conclusion as mentioned here -> ABCs just makes more sense if you have a common method or two (to keep your code DRY) :)
You might do a function consuming an instance of protocol instead of creating abc class with method
@@anatagenki2056 you mean using an object instantiated from a class that adheres to the protocol as argument for standalone function which I would normally define as method in ABC? Aren't you losing the abstraction of the function, when isolating it from protocol? Can you demonstrate on example?
ABCs are literally the thing that make abstraction and generic interfaces work. Like List doesn't do anything, it just defines what could be done and that it's of Generic type.
@@tajomnynominant hope formatting isn't corrupted. Let's assume you have an abstract class with one abstract method and one implemented: class Abstract(ABC): @abstractmethod def get_value(self) -> Any: pass def process_value(self) -> None: print(self.get_value()) class AbstractImpl(Abstract): def get_value(self) -> int: return 1 Same thing but using protocols: class Abstract(Protocol): def get_value(self) -> Any: pass def process_value(a: Abstract) -> None: print(a.get_value()) class AbstractImpl: def get_value(self) -> int: return 1 So the only difference is `process_value` is not accessible from `Abstract` namespace but from module scope. But I believe it is ok if you have a good design of project
Hey ... I have a question for the arjan with beard... How do you handle the dependency conflict, like I have a code base where a modules version is upgraded and then the whole system fails due to single dependency upgrade
Use virtualization. Find a working package version configuration and put it inside a venv / Docker container. This buys you time to fix the actual issue causing the version conflict. Hope this helps.
@@marcotroster8247 sure .. I'll give it a shot
So... to put this into more Object Oriented terms - Python's Protocol is more like operator/method overloads in C, C++, or C# (sad that Java doesn't do it, but oh well). And ABCs are... well... ABCs. Their name says it all. But Protocols are more like "if something shares my signature, then it's usable as a variable wherever I'm defined. Edit - Protocols are also like type extensions in the C language, where you can define a method is a static class and have its first argument be of the type you want to extend and then any object of that class can use that method.
I'd say ABC is more like overloading in C++, and Protocols are more like concepts/requires. I might be splitting hairs here though! I find Pythons type erasure like approach to be a double edged sword sometimes. It makes interfacing so much easier but puts a lot more emphasis on the caller being responsible for adhering to the contract.
Curious why you turned off all type checking (pylance, etc.) for this video? Protocols are meant to catch things at "compile" time rather than runtime, as you said. It would make no sense to use Protocols without turning type checking on haha
Yeah I've found it a bit concerning that he seems to be lacking even code colorization lately in vscode as if the language server isn't really parsing these scripts
Agreed! I might add for the newbies: using isinstance, and more in general relying on runtime type checking is the exact opposite of duck typing! Protocols are very good for composition, ABCs for inheritance
Thank you! I was really weirded out when he glossed over pylint and didn't mention mypy... From this video, there seems to be no good reason to use protocols. Without any mention of the reduced coupling and comparability advantages.
Teachers don’t introduce concepts with tools that are optional. What if someone doesn’t use an IDE that has these features? It’s foundational to understand concepts without all the nanny features.
@@frankclements6296 fair enough, but since Protocols are part of the typing python subsystem, it only makes sense to use it if you do static typing checks, be it in a IDE or with static checkers such as mypy...
Looks a lot like difference between abstract classes and interfaces in java. Maybe even more now after adding default methods. Of course there are differences, but looking at your conclusion it can be treated the same way.
So ABCs are more like Java's abstract classes or Scala's traits, where Protocols are more like Java's interfaces (with Manifold's @Structural annotation)
Yes, though ABCs can also used as interface, but so can Java's variants, I believe.
@@NostraDavid2 yup. Abstract classes can indeed also be used as an interface, though it does limit you to single inheritance. Scala traits however have no such limitation.
Uncle Bob is shaved... :) In the Java Restaurant you get a duck with the inscription “Class Duck”. And if you don't use the duck knife and duck fork you have to go to the toilet.
The world in which the restaurant exist doesn't even gets created in that case. 😀
You use more Protocols than ABCs? Seeing your videos that seems the case, but here you showed the caveat of protocol and analysing ABCs you just passed through. Do you prefer Protocols to avoid the inheritance mess it can become if you use nested inheritance?
Protocols are genius because you specify them at the place of usage. This allows you to trim the actual part of the interface you need and redefine it at each place of usage differently. It's very easy to mock unit tests with protocols. ABCs are trash because they introduce coupling whereas protocols don't. Embrace dynamic typing and functional programming techniques. It's Python, not Java.
dat DS9 reference!
It's easy to understand the difference between those. ABC is more like the classical OOP approach with "inheritance/abstract class". The protocol is more like the "trait implementation" Rust approach.
08:28, you can't do type checking with Protocol. That's enough to not use Protocol anywhere
Would love to know what kind of camera and microphone you use for your videos.
Please search in Arjan's channel, he already made a video about that in the early days of his channel. Please note however, that Arjan has changed (having a beard now) and the setup has probably changed since then as well.
@@jurgenrusch4041 thanks that’s helpful 🙄
We should absolutely stop eating at Python restaurants, because due to the speed of development in Python, that restaurant could've been a dental clinic yesterday.
It's nice to see that weird programmer humor isn't restricted to the US.
use abc to enforce a structure use protocol to enforce a contract
EVERYTHING is better with DS9 in it!
TOOWTDI was never meant to be serious but protocols seem like a violation of the spirit.
If you walk like a duck and quack like a duck, are you a duck? Yeah, let's not use ducktyping
Getting 404 when trying download design guide
Should be fixed now! Thank you for letting me know.
This one is harder to understand than the previous one. Just to note.
I don't like pylint because it's default kinda sucks and feels like it's made for Java. I don't want to write a docstring for each freaking .py file I make! I don't want to fight my tools every time I fart or sigh. I just use isort, black, flake8 and mypy. Ruff has my interest, but I'm waiting until 1.0 has released, or until the features being implemented have slowed down a little.
Your video shows all the waveat of using Protocol instead of ABC (no type checking, possible hard to debug errors, the fact that Protocol are not explicit). And you finish your video by saying that you use more Protocol than ABC. Why do you prefer Protocol over ABC then?
I'm only subscribed for your beard.
Python keep breaking it's own zen with no reason
You have to eat pythons in Python restaurant, not ducks !
both solutions are "meh, could have been better, but no..". Too much overhead to get the basics right
That thing with the Writable Protocol being okay with a write method regardless of what types the write method expects is honestly pretty bad. No other language with similar ideas (C++ concepts, Rust traits, Haskell typeclasses etc) has this problem, and IMO it makes the Python implementation worse than useless.
One thing not shown in the video is whether type checkers like mypy can detect this issue. If type checkers can detect this, then I think this is fine because Python type hinting are mostly designed for checking type safety before actually running the code?
@@NicolasChanCSY It's better in that case (is it the case?), yeah, but python making these weird tradeoffs for performance is honestly just wrong. If you want performance you should've picked a different language.
@@isodoubIet I understand that Python is not known for its performance/speed. But may you elaborate a bit what you mean "tradeoffs for performance"? To my understanding, type hinting is not about performance in Python (as least currently. Mojo uses them though), but about better tooling of maintaining good type consistency in the code. It was designed in such way to allow devs to progressively add typing into their codebase.
@@NicolasChanCSY What I mean is that (AFAIK) type hints are not checked by the interpreter for performance reasons, which is misguided IMO. I can understand type hints being ignored if the information to verify them is not available (keeping in line with the "gradual typing" design goal) but I don't think it makes any sense for the interpreter to accept code that can be determined to disagree with the annotations.
@@isodoubIetGuess you cannot compare a dynamic scripting language with a statically typed compiled language. For Python, it makes a lot of sense to specify protocols at the place of usage. Also the basic VSCode Python extension can detect type issues right away. Protocols are actually a really nice way to mock unit tests.
Never trust dudes without beards!