Page MenuHomePhabricator

Add solid subset rendering via large search bar, empty state for large DAGs
ClosedPublic

Authored by bengotow on Dec 6 2019, 1:13 AM.

Details

Summary

This diff adds a large bar to the bottom of the Explore UI that allows you to type one or more solid subset expressions (eg: ++solidname or solidname+*) and provides autocomplete to full solid names. The viewer renders only the subset described, making it much easier to work with large DAGs. It also adds an empty state that explains how to use the filter tool and expands on the previous iteration of the filtering logic to support expansion operators that specify recursion depth (+ vs ++ vs +++)

This diff also incorporates other cleanup:

  • We now center the viewport on solids you select
  • We now collapse the "Modes" selector in the sidebar by default for performance reasons
  • You can now copy-paste solid names and the zero-width spaces we insert to optimize display are removed
Test Plan

Run tests

Diff Detail

Repository
R1 dagster
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

bengotow created this revision.Dec 6 2019, 1:13 AM
bengotow updated this revision to Diff 7527.Dec 10 2019, 11:10 PM

Add new bar, put highlighting back, polish

bengotow retitled this revision from Replace solid filter highlighting with solid filter re-layout to Add solid subset rendering via large search bar, empty state for large DAGs.Dec 10 2019, 11:13 PM
bengotow edited the summary of this revision. (Show Details)
bengotow added a reviewer: alangenfeld.
bengotow updated this revision to Diff 7529.Dec 10 2019, 11:16 PM

Update changelog

bengotow edited the summary of this revision. (Show Details)Dec 10 2019, 11:23 PM
bengotow added inline comments.Dec 10 2019, 11:25 PM
js_modules/dagit/src/SidebarPipelineInfo.tsx
81

Fairly big perf boost on the airline demo pipeline right here actually...

This is overall excellent:

A couple comments while I was clicking around

  • When you press * the entire UI locks and then * doesn't not render. At least if you render the * it will feel less broken. Long term fix is web worker, as you mentioned. But some debouncing or something on the * would be good so the user gets a chance to activate the typeahead.
  • need to clear the search when navigating into a composite
  • might want to disable it in the composite solid view

For the single solid case, it would be good to keep the "focus" of the expression highlighted

bengotow updated this revision to Diff 7558.Dec 11 2019, 12:29 AM
  • Turn off this new feature in composites, since we are unable to save state at each layer and it was causing you to pop "up" to an empty view.
  • Mark the root nodes as the DAG's "focus" and highlight them in blue for better visualization
bengotow edited the summary of this revision. (Show Details)Dec 11 2019, 12:29 AM
schrockn requested changes to this revision.Dec 11 2019, 12:38 AM
  • We've lost to ability to search for single solid while having the full pipeline in context. I think we should re-add a solid typeahead on the left hand side and scope it to the current subselection in the center bar.
  • We should match on substring in the center bar but do autocomplete to the full solid name.
This revision now requires changes to proceed.Dec 11 2019, 12:38 AM
bengotow updated this revision to Diff 7570.Dec 11 2019, 4:46 PM
  • Put the solid jump bar back
  • Perform substring matching in solid queries, still autocomplete to full names
schrockn requested changes to this revision.EditedDec 11 2019, 9:34 PM

popping back to @bengotow pending addressing feedback in slack.

This revision now requires changes to proceed.Dec 11 2019, 9:34 PM
bengotow updated this revision to Diff 7697.Dec 13 2019, 8:07 PM
  • Perform substring matching in solid queries, still autocomplete to full names
  • Move the filter bar to the bottom
  • Automatically center the selected solid at all times
bengotow edited the summary of this revision. (Show Details)Dec 13 2019, 8:07 PM
bengotow edited the summary of this revision. (Show Details)Dec 13 2019, 8:09 PM
schrockn added a subscriber: prha.Dec 13 2019, 9:27 PM

This feels great. I'm especially excited about it in concert with @prha 's web worker stuff.

I'm going to resign to let prha or alex review the actual js. But from product perspectie this is great.

One thing I do thing this exposes the need for is a zoom bar so you know what to do if everything is off screen.

schrockn resigned from this revision.Dec 13 2019, 9:27 PM
schrockn added a reviewer: prha.
alangenfeld accepted this revision.Dec 13 2019, 9:41 PM

++ zoom bar (in follow up)

CHANGES.md
3–9

rebase - there is a 0.7.0 section now - you should tag it with Upcoming since its not currently marked as such

js_modules/dagit/src/SolidQueryImpl.ts
61–63

probably worth constifying this number - i had to hunt around to find it

js_modules/dagit/src/graph/SVGViewport.tsx
131

clean this up

js_modules/dagit/src/index.tsx
19–25

confused

maybe a comment about *why*

This revision is now accepted and ready to land.Dec 13 2019, 9:41 PM
bengotow updated this revision to Diff 7702.Dec 13 2019, 10:03 PM
  • Address PR feedback

Pushing up fixes now, thanks for the review @alangenfeld!

js_modules/dagit/src/index.tsx
19–25

Ahh will do - this is a bit of an unfortunate hack but I did a bunch of Googling and I think this really is our best bet for now...

alangenfeld added inline comments.Dec 13 2019, 10:25 PM
CHANGES.md
3

add an (Upcoming) up here if you get a chance

bengotow updated this revision to Diff 7704.Dec 13 2019, 10:48 PM
  • Add "(Upcoming)"