Page MenuHomeElementl

[dagit] Add a polling query for Run logs
ClosedPublic

Authored by dish on Jun 3 2021, 5:23 PM.

Details

Summary

Branch LogsProvider to use a polling query in cases where the websocket is not open. The query uses after to retrieve slices of logs.

Depends on https://dagster.phacility.com/D8220.

Test Plan

Force query branch in code. Launch a run. Verify that polling occurs correctly, and that the Gantt and logs table update correctly as well. Verify that the polling terminates properly when the status reaches a completed state.

Diff Detail

Repository
R1 dagster
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

will do some manual QA here

js_modules/dagit/packages/core/src/runs/LogsProvider.tsx
231–232

drop these two here i think - these are not terminal states

This looks great to me! Only other thing that occurs to me is that we also show streaming logs when you open a step's stderr | stdout panel from the bottom half of the run details view... I wonder if we need a similar fork over there?

js_modules/dagit/packages/core/src/runs/LogsProvider.tsx
186

You know, I think we have the websocket debounced to avoid flooding React with a zillion small state changes, it might only update once a second even with the stream of logs... Think we could potentially drop this to 1s and remove the websocket version? 🙈 Might be a good thing to investigate for a follow-up diff. Still preferable I think, but if we have after and this query is fast, maybe not by much!

This revision is now accepted and ready to land.Jun 3 2021, 6:59 PM

Use hook to determine global websocket availability, etc.

This revision was automatically updated to reflect the committed changes.