- Use pendulum datetimes, which have much better timezone and DST handling.
- Associate a timezone string with the scheduler, do the cron math using that timezone (and then convert back to UTC for the ticks)
Details
Details
- Reviewers
sashank alangenfeld max - Commits
- R1:fb652b6a321f: Make scheduler timezone-aware
Existing BK tests cover a scheduler in UTC. Next diff will add coverage for non-UTC timezones and daylight savings time
Diff Detail
Diff Detail
- Repository
- R1 dagster
- Branch
- tz3
- Lint
Lint Passed - Unit
No Test Coverage
Event Timeline
| python_modules/dagster/dagster/grpc/types.py | ||
|---|---|---|
| 207–223 | not that this is something we are worried about yet - but this is going to cause a version incompatibility problem right? | |
| python_modules/dagster/dagster/grpc/types.py | ||
|---|---|---|
| 207–223 | it is | |
| python_modules/dagster/dagster/grpc/types.py | ||
|---|---|---|
| 207–223 | I *think* all you would need to do make it compat is add =None on the new args, serdes already does a check to drop unknown args by keyword seems worth it since they are already opt_ | |
| python_modules/dagster/dagster/core/definitions/decorators/schedule.py | ||
|---|---|---|
| 253 | Oh wow this was just completely wrong before, wasn't it | |
| python_modules/dagster/dagster/scheduler/scheduler.py | ||
| 158 | Nit: We might want to have an error message here, so people know why this line is failing. Something along the lines of: In order to use the scheduler run command, you need to have DagsterCommandLineScheduler configured, but you have DagsterCronScheduler configured | |
| 189–190 | 👍 🎉 | |
| 189–190 | It might be helpful to add that example you showed me inline for future readers to know exactly what you're talking about | |
| 440 | Should we add another tag that's in the local execution time? To help people debug? | |
| python_modules/dagster/dagster/core/definitions/decorators/schedule.py | ||
|---|---|---|
| 253 | I don't think it was wrong when everything was in UTC - it may not even be wrong now, but its no longer needed to get the time exactly right as long as you're on the right day and will produce the right partition name | |
| python_modules/dagster/dagster/core/definitions/decorators/schedule.py | ||
|---|---|---|
| 253 | once you're no longer in UTC it's no longer true that, say, going back 3 hours at 3AM is no will always reach midnight, which you can count on in UTC | |