Page MenuHomeElementl

Introduce docker compose fixtures
ClosedPublic

Authored by jordansanders on Jul 2 2021, 8:10 PM.

Details

Summary

These pytest fixtures aim to replace much of what we currently do in
https://github.com/dagster-io/dagster/blob/c7cdcbc39d16f81287620f15edf42bc29eec5af0/.buildkite/dagster-buildkite/dagster_buildkite/utils.py#L71-L96
and https://github.com/dagster-io/dagster/blob/master/.buildkite/hooks/pre-exit

The two biggest benefits of this approach are:

  1. Conditional environment logic (i.e. different behavior when running on Buildkite) is relegated to the fixtures instead of living in the tests. While we'll still have two paths through every test that uses docker compose (one that uses localhost and one that jumps through some separate networking hoops), that branching will only live in a single place instead of repeating it over and over in each test.
  2. We can build more interesting test scenarios in pure Python. For example, with the old approach, if you wanted to do any setup before docker compose up, you needed to offload that test setup into a script and call it from the Buildkite step. Now, you can handle that test setup in pure Python inside your test fixtures.

There are two types of fixtures being introduced here:

  1. A higher level docker_compose fixture that's not customizable, but operates like a true Pytest fixture. You can use this if your docker-compose.yml lives in your test directory and if there's no additional setup or customization you need by simply including the fixture name as an argument to your test.
  2. A lower level docker_compose_cm fixture that yields a customizable contextmanager. This can be chained with other fixtures to produce highly customizable test setup.

These fixtures can be imported individually. Alternatively, as long as
dagster_test is installed, you can have test suites automatically
discover them via any of Pytest's plugin discovery options:

https://docs.pytest.org/en/6.2.x/writing_plugins.html#plugin-discovery-order-at-tool-startup

Initially, I'm setting the pytest_plugins variable in test suites that
use this. Eventually, I'd like to configure Buildkite and our
pyproject.toml to automatically include them in all of our tests - but
doing that requires some more intentional thought around _where_ these
fixtures should live. I see a few different options:

  • dagster_test - where I've chosen to place them initially. But I'm not sure we'd ever want this to be a dependency of _all_ libraries.
  • dagster - similar to where we put dagster.core.test_utils.py. But it seems odd to include pytest as a dependency of dagster.
  • A new library - perhaps dagster-pytest - that gets published to pypi. Although this is all specific to _our_ testing setup on Buildkite. A package with that name might be better as a customer-facing package with fixtures that allow you to easily test your own dagster code.
Test Plan

unit

Diff Detail

Repository
R1 dagster
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

Harbormaster returned this revision to the author for changes because remote builds failed.Jul 2 2021, 8:32 PM
Harbormaster failed remote builds in B33121: Diff 40822!

Always use absolute path so that we correctly infer the basename from the current working directory's docker-comopse.yml file

Harbormaster returned this revision to the author for changes because remote builds failed.Jul 2 2021, 9:04 PM
Harbormaster failed remote builds in B33138: Diff 40843!

Awesome! Could move description from the diff to code comments

python_modules/dagster-test/dagster_test/fixtures/docker_compose.py
108

So to use this, I'd call the docker_compose_cm with a fixed network name inside a fixture named network?

python_modules/dagster-test/dagster_test/fixtures/utils.py
22

oh nice, this is a default temp dir?

This revision is now accepted and ready to land.Jul 6 2021, 2:46 PM
python_modules/dagster-test/dagster_test/fixtures/docker_compose.py
108

Yup! Or not even strictly _inside_ a fixture named network - just from somewhere that your network has already been created (or will be created by the time docker-compose up finishes running.

So that could be inside a test that already relies on a network fixture. Or that could be from within a network fixture. Or that could be relying on the name of a network that's defined in the docker-compose.yml that you plan on using.

python_modules/dagster-test/dagster_test/fixtures/utils.py
22

It's actually the directory where the test was collected from.

So if you have:

test_foo/
  __init__.py
  test_foo.py
  test_bar/
    __init__.py
    test_bar.py
    docker-compose.yml

and you run pytest, then your cwd will obviously be wherever pytest is invoked from. request.fspath will be the path to the file where the test comes from. And test_directory will be the path to the directory where that file lives.

So for any test in test_bar.py, test_directory will return test_foo/test_bar which makes it easier for us to then identify test_foo/test_bar/docker-compose.yml

This revision was automatically updated to reflect the committed changes.