Page MenuHomePhabricator

Express schedule partition names and values in the timezone of the schedule
AbandonedPublic

Authored by dgibson on Oct 13 2020, 4:05 PM.

Details

Summary

Until this diff, the partition values (and range of partitions in the schedule) for schedules with timezones were being computed using UTC, which doesn't work for both usability reasons (why are my partitions in a weird offset?) and correctness reasons (yesterday's partition won't show up in the list until we pass midnight UTC).

This diff fixes that - but also means that a lot more things need to check state on the instance than they used to, and the function that generates a list of partitions from a partition set also needs to take in an instance.

Still adding a bit more testing but wanted to get an initial gut-check on this approach. Maintaining backwards compatibility with the cron scheduler is also a bit tricky since it doesn't consider timezones at all when generating partitions.

Test Plan

BK (Adding some more coverage now)

Diff Detail

Repository
R1 dagster
Branch
timezoneinstance4point5
Lint
Lint OK
Unit
No Unit Test Coverage

Event Timeline

Harbormaster returned this revision to the author for changes because remote builds failed.Oct 13 2020, 4:27 PM
Harbormaster failed remote builds in B19430: Diff 23611!
Harbormaster returned this revision to the author for changes because remote builds failed.Oct 13 2020, 5:07 PM
Harbormaster failed remote builds in B19433: Diff 23614!
Harbormaster returned this revision to the author for changes because remote builds failed.Oct 13 2020, 5:42 PM
Harbormaster failed remote builds in B19445: Diff 23627!
Harbormaster returned this revision to the author for changes because remote builds failed.Oct 13 2020, 6:19 PM
Harbormaster failed remote builds in B19447: Diff 23630!
dgibson added inline comments.
python_modules/dagster/dagster/core/instance/__init__.py
1079–1087

need to clean this up and ensure its fully back-compat with the legacy scheduler and the new scheduler