Page MenuHomePhabricator

Break pipeline explorer and sidebar queries apart, fetch only current solid layer
ClosedPublic

Authored by bengotow on Dec 4 2019, 10:38 PM.

Details

Summary
  • Only fetch the current layer of solidHandles
  • Only fetch "composite input mapping" info for the displayed parent solid
  • Only fetch the sidebar info when you click a solid to display it in the sidebar
Test Plan

Run fast!

Diff Detail

Repository
R1 dagster
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

bengotow created this revision.Dec 4 2019, 10:38 PM
alangenfeld requested changes to this revision.Dec 4 2019, 10:50 PM

fix lint

js_modules/dagit/src/PipelineExplorerRoot.tsx
33

do we actually want this behavior? when would our cache value be out of sync with the server?

js_modules/dagit/src/scripts/DownloadSchema.ts
18 ↗(On Diff #7104)

wat?

This revision now requires changes to proceed.Dec 4 2019, 10:50 PM

Will fix lint!

js_modules/dagit/src/PipelineExplorerRoot.tsx
33

That's a good point—this makes it so the page loads instantly when you go back to a previous page / pipeline, but it's true it may be out of sync for a second. Now that we're memoizing on the backend it might be fine to go network-only.

js_modules/dagit/src/scripts/DownloadSchema.ts
18 ↗(On Diff #7104)

Ahh since updating to Catalina my default shell is zsh and I've had to do this for some reason to avoid all python processes crashing with an Abort Trap 6...

bengotow updated this revision to Diff 7119.Dec 5 2019, 7:33 PM

Update for linting

alangenfeld accepted this revision.Dec 5 2019, 9:16 PM

once you get ts to pass

This revision is now accepted and ready to land.Dec 5 2019, 9:16 PM
bengotow updated this revision to Diff 7132.Dec 5 2019, 10:00 PM

Update data provided to our tests to match new root queries

bengotow updated this revision to Diff 7134.Dec 5 2019, 10:31 PM

Fix additional TS issue not found locally

Remaining build failure is in dagster-airflow, not related to this diff. Going to merge this.

This revision was landed with ongoing or failed builds.Dec 5 2019, 11:06 PM
This revision was automatically updated to reflect the committed changes.