Packaging Your Python Code With pyproject.toml | Complete Code Conversation
Learning how to package your code can seem overwhelming. This complete 52 minute conversation takes you through the steps one at a time, using modern best practices.
In this Code Conversation, you’ll follow a chat between Ian and Geir Arne demonstrating the relatively new, officially sanctioned way of setting up your Python projects using a pyproject.toml file and installing your package with pip. This offers nice benefits, such as:
- Being able to call your project from anywhere
- Playing on the same team as the import system
- Allowing for consistent imports
- Having one file that’ll work for many build systems
This is the complete conversation but the course is hosted here, if you want to learn more or find additional resources:
realpython.com/courses/packag...
And you'll learn how to:
- Structure files and folders in your project
- Understand different ways to run your script
- Explore how the import system works
- Explore the Python packaging world
- Write a pyproject.toml file to configure your package
- Install your pacakge with pip
I watch like 15 min of this video on the moment and I can say it is one of the best tutorials that i saw in a while. The way how you slowly divide problem and show all the casess, and explain why and how it is happening is brillant
I loved this one. All the basic questions I don't do were asked and answered with quality
timestamps: 1:40 running the first cli using python .cli.py 4:20 create directory "snakesay" to contain project. which will be the name of the package move all files to that folder 5:10 python snakesay does not work. need __main__. but still runs using .\snakesay\cli.py 6:48 rename clip.py to __main__ 7:30 python snakesay works now, but... 8:15 if you call a folder it needs to have __main__ 8:40 python -m to call module, does not work as python looks in its path 9:45 python uses two paths, the active path, and another path of all libraries. 13:45 absolute imports " from snakesay import snake" is needed when using -m ans points to snake module explicity 15:05 now "python snakesay" does not work again due to module not found 15:30 python with -m searches for different paths to get both "python" and "python -m" both working 16:00 you should not sys.path.append(yourpath) this will work, but does not scale well. this is commonly used but is a bad practice scaling problem is that you cant share it easily and does not help you merge programs 17:25 the plan is to install the package, you will pip install, but no need to upload it anywhere you can pip install your directory. 18:00 you will be able to use consistent inports and it will work in any CWD 19:00 historical solutions 20:24 intro to setuptools. it is not part of the standard library. 22:00 most importantant PEPs on how to setup packagages historically. messy history 22:50 create a local package 23:00 create pyproject.toml one directory up all consifg can be placed here, this replaces setup.py, or setup.cfg 24:20 setuptools home page shows config setup tools using .toml,build system, project, and optional dependenace, like a dictionary 25:50 PEP 621 storing metadata in pyproject.toml 26:40 toml is not python specific, it is general 27:05 copy the build-system section from the setuptools.pypa.io/en/latest/userguide/pyproject_config.html and paste into pyproject.toml 30:44 create "snakesay-project" and move contents into it 31:40 and move the .toml into the project folder and now install this into a virtual environment 32:00 in the project folder python -m venv venv 32:20 for windows activate the venv by ".\venv\Scripts\activate 32:50 in the toml create the [project] name = "snakesay" version ="1.0.0" 33:30 inside the snake-project folder python -m pip install -e . to allow changes updates to the source code 36:20 "Successfull installed snakesay-1.0.0" now can can call it from anywhere using with or without the -m flag 37:50 now you can "import snakesay" in any python script when the venv is activated 38:20 now there is no reason to think about the path, or relative imports,even if deep inside you package, you can always "from snakesave import snake" 38:48 the last step, to create an entry point, so that it is recognized by the terminal. 39:20 including project scripts to create an entry point, updating the .toml 41:50 need to pip instal the pyproject as changes where made to it. 42:30 now you can go anywhere on the commange line and call your program, but now it calls the program twice 43:40 to fix you need if __name__== "__main__" in the __main__.py 4920 Flint, Poetry support .toml as well now 46:35 whats an .egg? egg is a predecceor for wheels, it contains metadata and source for editable installs. 48:10 from inportlib import metdata to see metadata on a project from an egg. gitignore egg files 49:35 summary
thank you!
I've been a huge fan of the Real Python blog for ages but this is the first video I've seen. Phenomenal video Ian and Geir! So clear, so thorough. I too had been contemplating doing the "simple" path hack, but I figured it's worth investing time in learning the proper way to do things, and this lesson did not disappoint. It has already saved me hours of headache. Cheers.
Just brilliant. The whole back and forth conversations, step by step build up. its a gem of tutorial for any python developer . i learned alot just by this video. Thank you for this great effort!!!!!!!
This is just the right balance of technology description and teaching! Amazing work.
This is really a good review of pyproject.toml. I really appreciate this calm, slow introduction to the subject. Too many tutorials rush through the topic quickly, but here the understanding is built up slowly.
Conversation Style of explaining the Topic is an awesome Idea at the first place. Thank You both of You
Thank you for explaining the problem while also showing the solution.
this was a great tutorial. very great step-by-step explanations and intuitions.
What a brilliant tutorial. Makes what could be a complex topic simple and easy to understand.
God bless you two! I have been struggling with this for a very long time. I am so happy that I found this video and so thankful that you two created this video and explained everything in such an awesome and understandable way.
such a nice conversation and will not say a tutorial video but a tutorial should be like this.
Lol I've been having issues with this for a while and you guys explained it with just enough detail. Thank you
Finally a good explanation about packages! thank you!
I was struggling with python to do packaging right now, but you helped me! Thank you! Amazing vide 🤘
Every question I had was asked and answered in 1 video. Thanks a lot!
This video just made my life a lot better! Great thorough explanations, thank you!
Fantastic video, thank you !
You guys are awesome! I finally got it. What an amazing teaching!
Thank you so much for this video. This was awesome!
loved it, great tutorial!
great tutorial and was fun through out the video, thanks!
Thank you for the excellent tutorial. While AFAIK a package should have a __init__.py file in it, snakesay doesn't have one. It still works. So is the init file necessary? Thank you.
Thank you 🎉
Very nice videos answered a lot of questions which is in my mind for a long time .
Thanks for the video, was helpful.
THANK YOU!
Excellent tutorial! I was getting tired of being frustrated with all the 'modules not found', 'no parent package found errors'.
Great video!
Thanks!
Fantastic tutorial! Cheers!
Really helpful.
Excellent.
This is very well done, but having some chapters defined would be nice
Thanks
While I appreciate the forward looking nature of this conversation, it was disappointing that no mention was made of how to expect support for pyproject.toml to map with older versions of things. For example, my home setup (Xubuntu 20.04) has Python 3.8 and my workplace is still on Python 3.6. I assumed I could just look up those facts but they're proving non-obvious to find. I try to write Python code avoiding new "just for convenience" features so that my code can run on older versions - but that would be pointless if the packaging system I go to use requires newer versions. Or is this matter independent of the Python versions and instead reliant on versions of pip and setuptools being sufficiently updated? If so, I can see that the "current" version of pip requires CPython 3.7 or later (which begs more checking re Python 3.6). I can't yet find equivalent requirements for setuptools.
It works but it says command not found when i execute it with sudo and the script name.Anyfix would be deeply appreciated?
While viewing __main__.py in vscode, I notice that in the line 'from snakesay import snake', vscode does not recognize 'snakesay' (the text color is white). Is there something that needs to be done so vscode recognizes it, gives it the proper highlight, and doesn't have pylance show it as an error?
I have traditionally used extraPaths in .vscode to help with this. Kinda hacky, though. Would love to find a more automatic solution
Great tutorial, but the conversation is quite slow with long pauses
So how do I take this and put into a Docker container?
Was also wondering about this. I'm thinking this project could be stored in a repo, then you could clone the repo and add the package to requirements. Not sure if that would work or, if it does, if that's the best way to do it
8:43 Why to call with `-m` option?
that's how you run a script as module
8:20 python module
you should have explained how to run this script from anywhere in the system without having your virtualenv activated.
That kind of global scope is discouraged for Python packages. They note this when discussing installation
Thanks!
Thanks!