Page MenuHomePhabricator

Split airflow tests
ClosedPublic

Authored by catherinewu on Wed, May 13, 9:24 PM.

Details

Summary

Split airflow tests into 1. those that require K8s 2. those that require airflow db (but not k8s) 3. [default] those that do not require airflow db (nor k8s). This is to 1. make the buildkite runs faster and 2. to make sure airflow ingest works without an airflow db

https://github.com/dagster-io/dagster/issues/2449

Test Plan

Run tests

Diff Detail

Repository
R1 dagster
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

catherinewu created this revision.Wed, May 13, 9:24 PM
Harbormaster returned this revision to the author for changes because remote builds failed.Wed, May 13, 9:25 PM
Harbormaster failed remote builds in B11333: Diff 13927!
catherinewu requested review of this revision.Wed, May 13, 10:06 PM
catherinewu edited the summary of this revision. (Show Details)Thu, May 14, 6:34 AM
catherinewu edited the test plan for this revision. (Show Details)
catherinewu edited the summary of this revision. (Show Details)Thu, May 14, 6:37 AM
catherinewu added a reviewer: nate.
catherinewu edited the summary of this revision. (Show Details)Thu, May 14, 7:19 AM
catherinewu added a reviewer: nate.
nate accepted this revision.Thu, May 14, 6:00 PM

Cool LGTM, see one comment about pipeline.py

.buildkite/pipeline.py
497

maybe just push these down into airflow_tests()? seems like it'd be nice to keep the one top-level thing here, and then have an additional outer loop in the airflow_tests fn

python_modules/libraries/dagster-airflow/tox.ini
2

this is fine for now, but definitely starting to get towards the point where it might make sense to have multiple tox files

This revision is now accepted and ready to land.Thu, May 14, 6:00 PM
catherinewu edited the summary of this revision. (Show Details)Thu, May 14, 6:47 PM
This revision was automatically updated to reflect the committed changes.