Poetry: build.py example

Poetry contains an undocumented feature which allows running custom code when poetry build is executed. This can be particularly useful to, for example, compile a C extension, or perform some sort of pre-processing. This is done via a build.py and some configuration.

As can be seen by reading this thread, there has been a lot of churn over how to use this feature¹. Given that it’s undocumented, this comes with all the usual caveats regarding stability. But for now it appears to be the only way to call custom code during poetry build, which for some cases is a necessity.

So, without further delay, following is an example of using this feature, as it works today.

Example §

1. Add a build.py in the same directory as your pyproject.toml. Inside, put a def build(setup_kwargs): function, which is where your custom code will go:

build.py §

def build(setup_kwargs):
    print("Put your build code here!")

2. Delete your setup.py (assuming you already just have a shim).
3. Update your pyproject.toml to enable use of the build.py. The generate-setup-file = true tells Poetry to generate a setup.py when poetry build is run (hence the need to remove your own one), and the script = "build.py" causes the generated setup.py to have a line that executes your build.py:²

[tool.poetry.build]
script = "build.py"
generate-setup-file = true

4. Update your pyproject.toml to add setuptools as a build-system dependency:³

[build-system]
requires = ["poetry-core", "setuptools"]

5. Test your build:

$ poetry build
Put your build code here!
running build
running build_py

6. That’s it!

Caveat §

Beware that one caveat of this approach is that any use of a build.py script causes Poetry to assume build-specific tags are necessary, and it uses the most specific ones. In other words, if you are building under Python 3.10 using Linux, your wheel will now have a name such as:

my_package-1.2.3-cp310-cp310-manylinux_2_35_x86_64.whl

Rather than:

my_package-1.2.3-py3-none-any.whl

Since the use of build.py is undocumented, it seems that features related to this are low priority.⁴

Some workarounds have been suggested by the community:

• One fix suggested by tedivm is to use this script to convert the wheel into a pure Python wheel after the Poetry build has finished.
• A similar fix suggested by domvwt is to use this script.
• A different workaround suggested by Kaiser1989 is to call setuptools within Poetry to explicitly generate a wheel, though I was not able to get this to work.
• A suggestion by gitonthescene is to perform the build in two passes: first with your build.py enabled, and second with it disabled.

Two pass build §

I successfully used the two pass build suggestion. A simplified example implementation follows:

pyproject.toml §

# Your full pyproject.toml, including this section:
[tool.poetry.build]
script = "build.py"
generate-setup-file = true

pyproject.py3-none-any.toml §

# Same file as pyproject.toml, but *without* the [tool.poetry.build] section

Performing the build:

$ # First pass: generate the platform-specific build, including execution of
$ # your "build.py" to run custom code during the build.
$ poetry build

$ ls dist/
my_package-1.2.3-cp310-cp310-manylinux_2_35_x86_64.whl

$ # Second pass: generate the "pure Python" build.
$ mv pyproject.py3-none-any.toml pyproject.toml
$ poetry build

$ ls dist/
my_package-1.2.3-cp310-cp310-manylinux_2_35_x86_64.whl
my_package-1.2.3-py3-none-any.whl

This is not ideal, but if you are using tox/nox to perform the build, then these steps can be wrapped up in a single command.

Conclusion §

Ultimately, I decided not to use build.py. Since I am already wrapping by call to poetry build with Tox, I decided to put the custom build command in the Tox script instead, and avoid the complexity and potential instability of using this undocumented feature.

To do so, I created my_build_script.py: a simple standard library only Python script which performs the custom build command:

#!/usr/bin/env python3

import pathlib
import subprocess
# ...etc...

if __name__ == "__main__":
    # Do custom build...

Then in tox.ini called the script like so:

[testenv:py310-build]
allowlist_externals =
    poetry
    ./my_build_script.py
commands =
    ./my_build_script.py
    poetry build

Then I can run tox -e py310-build to perform my custom build.

Troubleshooting §

The build.py is not being called §

Check the output of poetry build to see whether it contains the following:

A setup.py file already exists. Using it.

If so, the issue is likely that you have generate-setup-file = true, but because a setup.py already exists, poetry is not generating one. The generated setup.py has a block of code that calls build.py. A fix is to delete your setup.py.

If that is not a suitable solution, you will probably need to modify your setup.py to call your custom build code.

ModuleNotFoundError: No module named ‘setuptools’ §

This should be fixed by adding setuptools to the build-system.

Backlinks §

• I’ve linked to this page from this GitHub comment.