Sam Hooke

Poetry: Running Black and isort with pre-commit hooks

All notes in this series:

Git Hooks provide “a way to fire off custom scripts when certain important actions occur”, and are typically stored in .git/hooks within your Git repository. For example, the Git pre-commit hook runs just before files are committed (e.g. when you run git commit), and so is a useful time to perform linting.

Related is a tool with the same name: pre-commit. This tool expects there to be a .pre-commit-config.yaml configuration file in the root of your Git repository. After cloning a Git repository, developers must then run pre-commit install: this invokes the pre-commit tool, which then writes the Git pre-commit hooks to .git/hooks. To try and help distinguish between the Git hook and the tool, I will use the phrase pre-commit tool whenever talking about the tool.

Using the pre-commit tool is particularly useful because “client-side hooks are not copied when you clone a repository”1. The tool introduces the .pre-commit-config.yaml file, which should be commited into the root of the Git repo, so it will be included in a clone2. Then, all the developer has to do is run pre-commit install3, assuming that the pre-commit tool is already installed.

Another significant advantage of the pre-commit tool is that, by default, it passes a list of only the files that have changed into your pre-commit hook entrypoint. In other words, if you use the pre-commit tool to run a linter, that linter will only check the files that have changed. This can be a big time saver, since it avoids needlessly linting all the files in your repository each time you commit.

In these notes we will install the pre-commit tool for a Python project using Poetry, and use pre-commit to automatically run black (to format code) and isort (to order imports). Typically, the .pre-commit-config.yaml would list each linting package, including the upstream location and version number, so that the tool can install these for you automatically. To avoid duplicating dependencies by listing them both in .pre-commit-config.yaml and pyproject.toml, we will integrate the pre-commit tool closely with Poetry.4

Install development dependencies §

The official documentation for pre-commit provides steps for installing it system-wide, but since it is just a Python package, we can install it as a development dependency within Poetry. This simplifies the setup process for other developers.

Since black and isort will be used by pre-commit, we will install them as development dependencies as well:

poetry add -G dev pre-commit black isort

To create/update the Poetry virtual environment with the development dependencies, run:

poetry install --with dev

Create .pre-commit-config.yaml §

Create a .pre-commit-config.yaml which runs Black using Poetry.5

.pre-commit-config.yaml §
fail_fast: true
repos:
- repo: local
  hooks:
    - id: black
      name: black
      entry: poetry run black
      language: system
      types: [file, python]

Going through each of these lines in detail:

fail_fast: true
This tells the pre-commit tool to stop after the first failure.
repos:
Begins the list of “repository mapping[s]”, i.e. where to get the code from for each pre-commit command.
- repo: local
Typically, this would specify the URL for the Git repository required for this hook. But the special keyword local can be used when the required code is provided in or by the local repository. In our case, the Poetry virtual environment will provide the required code.
hooks:
Begins a list of hooks provided by this repo.
- id: black
The ID of this hook. Seems to be arbitrary but let’s go with black to reflect the command being run.
name: black
The display name of this hook, as it appears in the terminal when the hook runs.
entry: poetry run black
The command to run. We use poetry run ... to run the command within the Poetry virtual environment. This calls black which executes the Black package, installed within the Poetry virtual environment. Note that we do not pass in any directories or files to include/exclude in the filter. As mentioned earlier, the pre-commit tool automatically passes in the list of files that have changed to our entrypoint. (This can be disabled with pass_filenames: false if necessary).
language: system
This specifies the programming language required for running this hook, of which many are supported. Since our hook does not call any specific language, but just expects the system to provide the poetry command, we use the system option.
types: [file, python]
This filters the files that are automatically passed into the entrypoint, and using types allows us to filter this list by file type. Specifically, the [file, python] type filter means that only Python files are included (e.g. those ending in *.py). It is possible to write a “raw” regex filter using files and exclude, but for most cases using types is sufficient and also less error-prone.

Install the pre-commit hooks §

Now that we have our configuration, first we must add it to Git, and then we can install the pre-commit hooks.

git add .pre-commit-config.yaml
poetry run pre-commit install

Now, whenever you do git commit ..., it will run the poetry run python -m black ... command like this:

$ git commit -m "Test commit"
black....................................................................Passed
[test-branch a4b13de] Test commit
 3 file changed, 2 insertions(+), 3 deletions(-)
 create mode 100644 .pre-commit-config.yaml

Extend configuration to support isort §

We can easily extend our .pre-commit-config.yaml to add isort as a hook:

.pre-commit-config.yaml §
fail_fast: true
repos:
- repo: local
  hooks:
    - id: black
      name: black
      entry: poetry run black
      language: system
      types: [file, python]
    - id: isort
      name: isort
      entry: poetry run isort
      language: system
      types: [file, python]

Since isort is “provided” by the local repo, we do not need to add another repo, but can add it as a second hook within the same repo.

By default, isort and black will “fight” as their default configuration disagrees on how to handle spacing between imports. To make them play nicely, you must enable the black profile in isort by updating your pyproject.toml with the following:

pyproject.toml §
[tool.isort]
profile = "black"

For further isort configuration suggestions, see this note.

Advanced usage §

Normally you always want the pre-commit hooks to run when committing, but in some circumstances you might want to do one without the other. These two scenarios are covered below:

Running the pre-commit hooks without committing §

To run the pre-commit tool hooks without actually comitting anything to Git, use the pre-commit run command within Poetry. If you have not made any changes then you will get a (no files to check)Skipped message, because no files are being passed in. This is useful to verify that the pre-commit tool is indeed only passing in files that have changed:

$ poetry run pre-commit run
black................................................(no files to check)Skipped
isort................................................(no files to check)Skipped

If you want to run the pre-commit tool hooks on all files, regardless of whether they have changed, use the --all-files option:

$ poetry run pre-commit run --all-files
black....................................................................Passed
isort....................................................................Passed

Committing without running the pre-commit hooks §

To commit something to Git without any of the pre-commit hooks running, do:

git commit --no-verify

The --no-verify options tells Git not to invoke anything in .git/hooks.

  1. Quote from the official Git documentation here. Note that server-side Git Hooks are also a thing, but this note only covers client-side Git Hooks. ↩︎

  2. This 13 year old Stack Overflow question about whether it is considered “bad practice” to store the client-side Git Hooks inside the repository has ten answers, all of which effectively say that it is okay, or even “good practice” to include the Git Hooks in some way. Three of the answers (those written or updated since about 2020) point towards using the pre-commit tool, which is covered in this note. ↩︎

  3. Apparently, this can be simplified even further. By using this poetry-pre-commit-plugin, the pre-commit install command can be run automatically for projects using Poetry. I have not yet tried this plugin out, because I wanted to focus on getting the basics working first, but it could be worth enabling in future. ↩︎

  4. An alternative approach is to use Sync with Poetry, a pre-commit hook which automatically updates your .pre-commit-config.yaml whenever you commit corresponding changes to your pyproject.toml. For example, if you bump the version of black in pyproject.toml, when you commit, this hook will update the version of black you have specified in .pre-commit-config.yaml. I did not take this approach because I wanted to keep everything running within the Poetry virtual environment if possible. ↩︎

  5. Thank you to this config from this article for getting me started with this information. ↩︎

All notes in this series: