Python - Poetry – JIHONGO's Blog

Preface

I read some articles discussing Poetry recently, and I noticed that pyproject.toml seems to be the future trend (PEP 518, PEP 621), so I find some time study it.

What is Poetry

According to Poetry GitHub

Poetry helps you declare, manage and install dependencies of Python projects, ensuring you have the right stack everywhere.

It supports Python 3.7+

Similar to pip, provides package management but more powerful functions than pip

Track/Build/Publish for packages
DEPENDENCY RESOLVER
ISOLATION
INTUITIVE CLI

Past Practices and Defects

I used pipenv as a virtual environment management tool for Python, but it is not good for package dependency resolution.

Supposing you need to remove a package, pipenv uninstall will only remove the specified package, and the dependency packages will not be removed.

You need to remove it manually, but you can not determine whether it is also depended on by other packages.

If you remove it casually, other packages may not work properly.

As kyo described in this article (zh-tw), if flask and black are used in the same project, removing flask and then manually removing click will cause black to not work properly.

Start Poetry journey

Install Poetry

Linux

1

curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python -

Windows powershell

1

(Invoke-WebRequest -Uri https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py -UseBasicParsing).Content | python -

For Windows, it will be installed in %APPDATA%\Python\Scripts, and this path will also be set to the environment variable PATH during installation.

Check the version of poetry

1

poetry --version

Upgrade the version of poetry

1

poetry self update

Check config of poetry

1

poetry config --list

The cache-dir is the the path to the cache directory used by Poetry.

If set virtualenvs.in-project to true, Poetry will create the virtualenv inside the project root.

1

poetry config virtualenvs.in-project true

However, at present, if in-project is set to true, there seems to be a BUG when using the poetry-command to remove the virtual environment and it cannot be removed normally, but we can still remove the virtual environment by manually deleting the folder.

Create new project

Create a new poetry project

1
2


poetry new poetry-demo
cd poetry-demo

The folder tree will be

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


poetry-demo
│  pyproject.toml
│  README.rst
│
├─poetry_demo
│      __init__.py
│
└─tests
        test_poetry_demo.py
        __init__.py

The pyproject.toml is the most important file

In addition, if there is a pre-existing project, you can create pyproject.toml via init

1
2


cd pre-existing-project
poetry init

And then we create virtual environment

1
2
3
4
5


poetry env use python

# There is a new .venv folder
Creating virtualenv poetry-demo in D:\Projects\Test\Python\poetry-demo\.venv
Using virtualenv: D:\Projects\Test\Python\poetry-demo\.venv

Then active the virtual environment (there must be pyproject.toml in the folder, otherwise an error will be occurred)

1

poetry shell

Some themes of Terminal supports hints if you are in a virtual environment

You can also use poetry env info to get current virtual environment information

1
2
3


poetry env info
poetry env info --path  # Get the path to the virtual environment
poetry env list # List all the virtual environments associated with the current project

Exit virtual environment

1

exit

Add packages

Use poetry add to add packages

1
2


poetry add requests
poetry add flake8 --dev # Install to dev-dependencies

pyproject.toml will be updated at the same time

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16


[tool.poetry]
name = "poetry-demo"
version = "0.1.0"
description = ""
authors = ["JIHONGO <jihongoidv@gmail.com>"]

[tool.poetry.dependencies]
python = "^3.9"
requests = "^2.27.1"    # requests we added just now

[tool.poetry.dev-dependencies]
pytest = "^5.2"

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

It can be observed from the above that other packages are also installed when requests is installed, such as certifi, urllib3, etc. These are dependencies of requests and will not be recorded in pyproject.toml

We can use the poetry show command to observe the package dependencies, which are derived from poetry.lock

1

poetry show --tree

If you want to remove the package, you need to use the poetry remove command

1

poetry remove LIBRARY_NAME

The removed part will not be demonstrated here, you can try to install the above mentioned flask and black libraries at the same time and then remove one of them to see what happens

Run

We can execute commands in the virtual environment through run

For example, the following command can list the version of Python being executed

1

poetry run python -V

Let’s create a simple script (main.py)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


poetry-demo
│  pyproject.toml
│  README.rst
│
├─poetry_demo
│      __init__.py
│      main.py  # new script
│
└─tests
        test_poetry_demo.py
        __init__.py

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14


# main.py
import sys


def start(args=sys.argv[1:]):
    print(f"Rinning main.py: start (args: {args})")


def main():
    print("Running main.py: main")


if __name__ == "__main__":
    main()

It can be executed via poetry run python

1
2
3
4


poetry run python .\poetry_demo\main.py

# Result
Running main.py: main

We can also execute commands by adding Script

Modify pyproject.toml, add tool.poetry.scripts

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22


[tool.poetry]
name = "poetry-demo"
version = "0.1.0"
description = ""
authors = ["JIHONGO <jihongoidv@gmail.com>"]

[tool.poetry.dependencies]
python = "^3.9"
requests = "^2.27.1"

[tool.poetry.dev-dependencies]
pytest = "^5.2"
flake8 = "^4.0.1"

# add tool.poetry.scripts
[tool.poetry.scripts]
main = "poetry_demo.main:main"
start = "poetry_demo.main:start"

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

Then we can execute these commands

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


poetry run main

# Result
Running main.py: main

# Run with args
poetry run start arg1 arg2

# Result
Rinning main.py: start (args: ['arg1', 'arg2'])

Output requirements.txt

If you need to output the package as requirements.txt, you can generate it with the following command

1

poetry export -f requirements.txt --output requirements.txt --without-hashes

Common commands

poetry new PROJECT_NAME
poetry init
poetry env use python
poetry shell
poetry add LIBRARY_NAME
poetry install
poetry remove LIBRARY_NAME
poetry show --tree
poetry export -f requirements.txt --output requirements.txt --without-hashes

Conclusion

The current version of poetry is about 1.1.13, and it seems that it is still being optimized. There may be some potential bugs, such as the previously mentioned problem that env cannot be removed. I hope it will get better and better in the future.

For me, I may substitute poetry for pipenv, although I still need to get familiar with it, but from all aspects, poetry seems to be the best solution and trend at present, of course I have to try it.

Python - Poetry