Page MenuHomePhabricator

Surface multiple errors in daemon
Needs ReviewPublic

Authored by johann on Wed, Jan 6, 8:10 PM.

Details

Reviewers
dgibson
prha
Summary

Allow daemons to yield non-fatal errors to be reported in daemon heartbeats.

Each heartbeat includes the full list of errors from the last iteration. If it used errors from the current generation, it could randomly exclude errors based on if the daemon had reached them yet.

TODO: graphql +dagit side

Test Plan

Unit

Diff Detail

Repository
R1 dagster
Branch
daemon-multiple-errors (branched from master)
Lint
Lint OK
Unit
No Unit Test Coverage

Event Timeline

Harbormaster returned this revision to the author for changes because remote builds failed.Wed, Jan 6, 8:27 PM
Harbormaster failed remote builds in B23765: Diff 28900!
Harbormaster returned this revision to the author for changes because remote builds failed.Wed, Jan 13, 9:09 PM
Harbormaster failed remote builds in B24310: Diff 29578!
johann edited the test plan for this revision. (Show Details)
johann added reviewers: dgibson, prha.
johann published this revision for review.Wed, Jan 13, 9:16 PM

Publishing early, tests will fail due to missing gql changes

python_modules/dagster/dagster/daemon/controller.py
118

This whole block is rather messy, happy to take suggestions. Maybe Dan's earlier proposal to iterate through daemons and remove them as they finish would help, I just need to remember python's rules about mutating the data under an iterator

wait for first iteration to report heartbeat