Page MenuHomePhabricator
Feed Advanced Search

Tue, Jan 21

bengotow updated the diff for D1860: Basic composite expansion in the Dagit explore tab.

Cleanup & Update tests

Tue, Jan 21, 10:10 PM
bengotow added inline comments to D1860: Basic composite expansion in the Dagit explore tab.
Tue, Jan 21, 10:04 PM
bengotow updated the summary of D1860: Basic composite expansion in the Dagit explore tab.
Tue, Jan 21, 10:00 PM
bengotow created D1860: Basic composite expansion in the Dagit explore tab.
Tue, Jan 21, 9:57 PM

Fri, Jan 17

bengotow added a comment to D1855: Gaant-style execution plan visualization behind feature flag.

Ahh @alangenfeld that's a good question—for the serial execution engine displaying the dependency graph isn't quite what we want. Could we make the execution plan to include the order it plans to execute steps? Right now the plan doesn't encode enough information for us to know what the true parallelism / timing will look like (at least I don't think?) We could detect the serial execution engine and show them in a waterfall, but we might get the order wrong I think?

Fri, Jan 17, 7:46 PM
bengotow updated the diff for D1855: Gaant-style execution plan visualization behind feature flag.
  • Use fewer large nested functions in Gaant layout code
Fri, Jan 17, 7:42 PM
bengotow added inline comments to D1855: Gaant-style execution plan visualization behind feature flag.
Fri, Jan 17, 6:31 PM
bengotow updated the diff for D1855: Gaant-style execution plan visualization behind feature flag.

Comment Gaant layout algo

Fri, Jan 17, 6:28 PM
bengotow updated the summary of D1855: Gaant-style execution plan visualization behind feature flag.
Fri, Jan 17, 6:11 PM
bengotow created D1855: Gaant-style execution plan visualization behind feature flag.
Fri, Jan 17, 6:10 PM

Mon, Jan 13

bengotow added a comment to D1798: Replace playground subset selector with one that leverages solid query syntax.

Hmm that's a good point re: partially invalid queries — it seems every portion of the query should be required to match at least one solid. Until we can display errors in context in the field, we should probably err on the side of it being valid though.

Mon, Jan 13, 4:55 PM
bengotow updated the diff for D1798: Replace playground subset selector with one that leverages solid query syntax.
  • Allow space as a delimiter in query syntax
  • Only allow user to submit dialog when the query is valid
  • Blur the active field when firing a shortcut so the query is committed
Mon, Jan 13, 3:16 PM
bengotow planned changes to D1798: Replace playground subset selector with one that leverages solid query syntax.
Mon, Jan 13, 2:45 PM

Wed, Jan 8

bengotow updated the summary of D1798: Replace playground subset selector with one that leverages solid query syntax.
Wed, Jan 8, 9:56 PM
bengotow created D1798: Replace playground subset selector with one that leverages solid query syntax.
Wed, Jan 8, 8:33 PM
bengotow added a comment to D1763: New top level navigation in Dagit, rework of pipeline scope.

Re: Is there somewhere on the screen where we can put the "Alt" hint for discovering shortcuts? Pretty hard to figure out. - let's try to put it on the new home page near the bottom, I think that screen will have an empty corner or two.

Wed, Jan 8, 7:28 PM
bengotow updated the diff for D1763: New top level navigation in Dagit, rework of pipeline scope.
  • Prevent shortcuts from re-rendering children on on/off
  • Merge branch 'master' of github.com:dagster-io/dagster into sfbg/layout-overhaul
Wed, Jan 8, 7:27 PM
bengotow added a comment to D1763: New top level navigation in Dagit, rework of pipeline scope.

Hey hey! Rebasing this now. I initially tried for Alt-Enter, but if a dialog appears letting you know that the run could not be started / config had errors, Alt-Enter also closes the dialog so it usually just flickers. I think we could bind the event to the keyup phase of the Alt-Enter shortcut, will look and see.

Wed, Jan 8, 6:58 PM

Tue, Jan 7

bengotow updated the diff for D1763: New top level navigation in Dagit, rework of pipeline scope.
  • Bind Alt-G to the Start button
  • Fix broken Alt-S binding in Explore tab
Tue, Jan 7, 9:29 PM
bengotow updated the diff for D1763: New top level navigation in Dagit, rework of pipeline scope.
  • Prevent shortcuts from re-rendering children on on/off
Tue, Jan 7, 9:13 PM
bengotow updated the diff for D1763: New top level navigation in Dagit, rework of pipeline scope.
  • Prevent shortcuts from sticking if you tab away from the window
Tue, Jan 7, 9:02 PM
bengotow added inline comments to D1788: (1/n run-storage) Allow multiple tags in PipelineRunsFilter.
Tue, Jan 7, 8:58 PM
bengotow accepted D1780: [dagit] add launch run button.

Ok sounds good to me then 😉

Tue, Jan 7, 8:55 PM
bengotow updated the diff for D1763: New top level navigation in Dagit, rework of pipeline scope.
  • Fix spelling
  • Switch shortcuts to Opt-S and Opt-P so they work when you’re in the text editor
  • Allow you to navigate via keyboard Alt-1, Alt-2, Alt-3 through left hand tabs
  • Schedule / Scheduler => Schedules
Tue, Jan 7, 8:44 PM
bengotow added a comment to D1780: [dagit] add launch run button.

This looks good to me, just added some comments and suggestions inline. Nothing super big!

Tue, Jan 7, 1:01 AM

Fri, Jan 3

bengotow resigned from D1773: [dagit] add cause to PythonError.

The client side of this LGTM! Will resign and let someone take a look at the python, which looks quite a bit above my pay grade 😅

Fri, Jan 3, 10:56 PM
bengotow requested changes to D1772: enable multi-tag run filtering.
Fri, Jan 3, 6:59 PM

Thu, Jan 2

bengotow planned changes to D1763: New top level navigation in Dagit, rework of pipeline scope.
Thu, Jan 2, 10:19 PM
bengotow added a comment to D1763: New top level navigation in Dagit, rework of pipeline scope.
I think the playground should build a url based on pipeline selected just like the pipelines tab.
Thu, Jan 2, 10:18 PM

Tue, Dec 31

bengotow updated the diff for D1763: New top level navigation in Dagit, rework of pipeline scope.

New config picker needs to pass pipeline name to new sessions

Tue, Dec 31, 5:11 PM
bengotow updated the diff for D1763: New top level navigation in Dagit, rework of pipeline scope.

Rebase

Tue, Dec 31, 5:48 AM
bengotow retitled D1763: New top level navigation in Dagit, rework of pipeline scope from [Pending self review] New top level navigation in Dagit, rework of pipeline scope to New top level navigation in Dagit, rework of pipeline scope.
Tue, Dec 31, 5:20 AM
bengotow updated the diff for D1763: New top level navigation in Dagit, rework of pipeline scope.
  • Cleanup of diff
Tue, Dec 31, 5:20 AM

Mon, Dec 30

bengotow updated the summary of D1763: New top level navigation in Dagit, rework of pipeline scope.
Mon, Dec 30, 11:45 PM
bengotow updated the summary of D1763: New top level navigation in Dagit, rework of pipeline scope.
Mon, Dec 30, 11:44 PM
bengotow retitled D1763: New top level navigation in Dagit, rework of pipeline scope from New top level navigation in Dagit, rework of pipeline scope to [Pending self review] New top level navigation in Dagit, rework of pipeline scope.
Mon, Dec 30, 11:41 PM
bengotow created D1763: New top level navigation in Dagit, rework of pipeline scope.
Mon, Dec 30, 11:40 PM

Dec 19 2019

bengotow retitled D1708: Make RunMetadataProvider less dependent on log ordering, fix dagit crash from Check for existence of expectationResults/materializations to Make RunMetadataProvider less dependent on log ordering, fix dagit crash.
Dec 19 2019, 6:27 PM
bengotow updated the diff for D1708: Make RunMetadataProvider less dependent on log ordering, fix dagit crash.
  • Remove immer
  • Update specs
Dec 19 2019, 5:46 PM
bengotow retitled D1708: Make RunMetadataProvider less dependent on log ordering, fix dagit crash from Check for existence of expectationResults/materializations to Update RunMetadataProvider to be less dependent on the event ordering / fix crash.
Dec 19 2019, 5:46 PM
bengotow updated the diff for D1708: Make RunMetadataProvider less dependent on log ordering, fix dagit crash.
  • Revert previous check for metadata presence
  • Update RunMetadataProvider to be less dependent on event ordering
Dec 19 2019, 5:42 PM
bengotow commandeered D1708: Make RunMetadataProvider less dependent on log ordering, fix dagit crash.
Dec 19 2019, 5:24 PM
bengotow added a comment to D1708: Make RunMetadataProvider less dependent on log ordering, fix dagit crash.

Hey @max! Ok - so I think that this is happening because we are receiving events out of order somehow, or otherwise missing the ExecutionStepStartEvent for a given step. It looks like if we receive a ExecutionStepFailureEvent with no preceding ExecutionStepStartEvent event, the execution plan step metadata (the object containing the materializations) gets initialized like this:

Dec 19 2019, 5:20 PM

Dec 18 2019

bengotow accepted D1698: Fix typo.

Ahh good catch! 👍

Dec 18 2019, 8:14 PM
bengotow accepted D1655: Partition selector in dagit config editor.

The JS side of this looks great to me—the only thing is that the introduction of state in the ConfigEditorConfigPicker might introduce some /slightly/ weird behavior because the state isn't saved per-tab. I think that if you switch tabs, the partition picker may remain present (or it may disappear and upon returning to a previous tab still be gone. I'm not sure which direction it will fail in offhand.)

Dec 18 2019, 12:09 AM

Dec 17 2019

bengotow updated the diff for D1693: Slightly increase solid size, allow names to wrap to two lines in DAG.

Only enable in non-minified mode

Dec 17 2019, 11:58 PM
bengotow planned changes to D1693: Slightly increase solid size, allow names to wrap to two lines in DAG.
Dec 17 2019, 11:50 PM
bengotow added a comment to D1693: Slightly increase solid size, allow names to wrap to two lines in DAG.

Sounds good! I agree re: zoomed out view - you get an extra word or two but even that typically gets clipped rendering it largely useless.

Dec 17 2019, 11:50 PM
bengotow added a comment to D1691: Add missing semicolons in CSS - styled-components no longer post-processing.

Hey @prha — I did a search for styled. and went through the whole project to check that we have the ; in place now. I think this should be exhaustive!

Dec 17 2019, 11:47 PM
bengotow updated the summary of D1693: Slightly increase solid size, allow names to wrap to two lines in DAG.
Dec 17 2019, 8:30 PM
bengotow created D1693: Slightly increase solid size, allow names to wrap to two lines in DAG.
Dec 17 2019, 8:29 PM
bengotow created D1691: Add missing semicolons in CSS - styled-components no longer post-processing.
Dec 17 2019, 7:15 PM

Dec 16 2019

bengotow accepted D1680: [dagit] solids page perf.

This looks great. My only comment is that it's a bit strange that the component for the right hand side of the page is called SolidSidebar since it's the larger portion of the screen. I know it sort of inherited that naming because it re-uses the SidebarSolidDefinition component, but maybe we could call it SolidDetailContainer or something like that...

Dec 16 2019, 9:26 PM

Dec 15 2019

bengotow accepted D1668: (new-config-parsing-3) innerTypes --> recursiveConfigTypes.

Good call—I think this naming is much more clear.

Dec 15 2019, 9:40 PM
bengotow accepted D1657: Add web worker to calculate dagre layout for large graphs.

I'd defer to @alangenfeld but this looks great! Approved but with a couple inline comments - would definitely be good to double check and make sure the caching is working the way we intend.

Dec 15 2019, 9:38 PM
bengotow updated the diff for D1663: Fix dagit tests, switch to smarter styled-components autogen classnames.

Rebase

Dec 15 2019, 8:59 PM

Dec 14 2019

bengotow updated the summary of D1663: Fix dagit tests, switch to smarter styled-components autogen classnames.
Dec 14 2019, 5:17 PM
bengotow created D1663: Fix dagit tests, switch to smarter styled-components autogen classnames.
Dec 14 2019, 5:16 PM
bengotow updated the diff for D1659: Add a zoom bar to the explorer, fix expanding composites showing empty viewport.

Fix tests

Dec 14 2019, 3:26 PM
bengotow created D1659: Add a zoom bar to the explorer, fix expanding composites showing empty viewport.
Dec 14 2019, 12:02 AM

Dec 13 2019

bengotow updated the diff for D1552: Add solid subset rendering via large search bar, empty state for large DAGs.
  • Add "(Upcoming)"
Dec 13 2019, 10:49 PM
bengotow added a comment to D1552: Add solid subset rendering via large search bar, empty state for large DAGs.

Pushing up fixes now, thanks for the review @alangenfeld!

Dec 13 2019, 10:04 PM
bengotow updated the diff for D1552: Add solid subset rendering via large search bar, empty state for large DAGs.
  • Address PR feedback
Dec 13 2019, 10:03 PM
bengotow accepted D1651: Extract layout-only methods into a standalone file that can be called from a web-worker (1 of 4).

This looks good 👍 I'm not sure how you're planning on structuring the webworker sources on the filesystem, but it might make sense to have a graph-worker folder (or something similar) for this dagre layout code so we can make it super clear that it's targeting a different JS context.

Dec 13 2019, 8:20 PM
bengotow accepted D1652: extract graph container from PipelineExplorer (2 of 4).
Dec 13 2019, 8:17 PM
bengotow added a comment to D1652: extract graph container from PipelineExplorer (2 of 4).

This looks great to me. I actually think we may want to double-down on this PipelineGraphContainer and move a bit more of the state down into it in a subsequent diff. I've been looking in to how we would "stack" viewports rather than replacing the viewport as you drill down into composites, and it's becoming clear that we need the state of each layer to be owned in a container rather than at the top. At any rate, seems like it'd build on this nicely.

Dec 13 2019, 8:16 PM
bengotow accepted D1654: Extract dom utils from Utils.

Looks good to me! I feel like every project grows until there's a DOMUtils / Utils / DebugUtils, etc, but this seems like the correct approach here 😂

Dec 13 2019, 8:13 PM
bengotow accepted D1656: Handle PythonErrors.

This looks good to me! I'm surprised this failed to build but you probably just need to run make generate-types.

Dec 13 2019, 8:12 PM
bengotow updated the summary of D1552: Add solid subset rendering via large search bar, empty state for large DAGs.
Dec 13 2019, 8:09 PM
bengotow updated the summary of D1552: Add solid subset rendering via large search bar, empty state for large DAGs.
Dec 13 2019, 8:07 PM
bengotow updated the diff for D1552: Add solid subset rendering via large search bar, empty state for large DAGs.
  • Perform substring matching in solid queries, still autocomplete to full names
  • Move the filter bar to the bottom
  • Automatically center the selected solid at all times
Dec 13 2019, 8:07 PM

Dec 11 2019

bengotow updated the diff for D1552: Add solid subset rendering via large search bar, empty state for large DAGs.
  • Put the solid jump bar back
  • Perform substring matching in solid queries, still autocomplete to full names
Dec 11 2019, 4:46 PM
bengotow updated the summary of D1552: Add solid subset rendering via large search bar, empty state for large DAGs.
Dec 11 2019, 12:29 AM
bengotow updated the diff for D1552: Add solid subset rendering via large search bar, empty state for large DAGs.
  • Turn off this new feature in composites, since we are unable to save state at each layer and it was causing you to pop "up" to an empty view.
  • Mark the root nodes as the DAG's "focus" and highlight them in blue for better visualization
Dec 11 2019, 12:29 AM

Dec 10 2019

bengotow accepted D1627: Refactor solid selector to defer query until selector is opened.

LGTM!

Dec 10 2019, 11:54 PM
bengotow added inline comments to D1627: Refactor solid selector to defer query until selector is opened.
Dec 10 2019, 11:42 PM
bengotow added inline comments to D1552: Add solid subset rendering via large search bar, empty state for large DAGs.
Dec 10 2019, 11:25 PM
bengotow updated the summary of D1552: Add solid subset rendering via large search bar, empty state for large DAGs.
Dec 10 2019, 11:23 PM
bengotow updated the diff for D1552: Add solid subset rendering via large search bar, empty state for large DAGs.

Update changelog

Dec 10 2019, 11:16 PM
bengotow retitled D1552: Add solid subset rendering via large search bar, empty state for large DAGs from Replace solid filter highlighting with solid filter re-layout to Add solid subset rendering via large search bar, empty state for large DAGs.
Dec 10 2019, 11:13 PM
bengotow updated the diff for D1552: Add solid subset rendering via large search bar, empty state for large DAGs.

Add new bar, put highlighting back, polish

Dec 10 2019, 11:10 PM
bengotow accepted D1600: Substantially reduce fetching required to get the entire schema.

Oh man this looks great to me. Really glad we were able to get innerTypes out of here and the new interfaces for the ConfigTypeSchema component look great. Ship it!

Dec 10 2019, 6:50 PM

Dec 6 2019

bengotow updated the diff for D1549: [dagit] Fix wrapping in sidebar, disable reflow of fan-out.

Removing weighting method

Dec 6 2019, 6:19 PM
bengotow added inline comments to D1549: [dagit] Fix wrapping in sidebar, disable reflow of fan-out.
Dec 6 2019, 6:16 PM
bengotow retitled D1549: [dagit] Fix wrapping in sidebar, disable reflow of fan-out from Weight DAG edges by solid name similarity, fix wrapping in sidebar to [dagit] Fix wrapping in sidebar, disable reflow of fan-out.
Dec 6 2019, 7:00 AM
bengotow updated the diff for D1549: [dagit] Fix wrapping in sidebar, disable reflow of fan-out.

Disable weighting based on name similarity, update tests

Dec 6 2019, 6:59 AM
bengotow planned changes to D1549: [dagit] Fix wrapping in sidebar, disable reflow of fan-out.

Yeah that's a good point—I'll omit the edge weighting for now so we can push in the rest of these fixes. Long term it'd be kind of nice if users could choose between a couple graph layout algorithms but I agree this approach may work only for their DAG.

Dec 6 2019, 6:51 AM
bengotow created D1552: Add solid subset rendering via large search bar, empty state for large DAGs.
Dec 6 2019, 1:13 AM
bengotow created D1550: Replace current two-step zoom with smooth zoom.
Dec 6 2019, 1:11 AM
bengotow updated the diff for D1549: [dagit] Fix wrapping in sidebar, disable reflow of fan-out.
  • Also fix issue with "black fill" appearing on external connection lines when viewing a composite (for unknown reason)
  • Turn off "max per row" code intended to wrap long lines of solids into blocks, does not work well on new example
Dec 6 2019, 12:11 AM

Dec 5 2019

bengotow updated the summary of D1549: [dagit] Fix wrapping in sidebar, disable reflow of fan-out.
Dec 5 2019, 11:39 PM
bengotow created D1549: [dagit] Fix wrapping in sidebar, disable reflow of fan-out.
Dec 5 2019, 11:39 PM
bengotow added a comment to D1540: Break pipeline explorer and sidebar queries apart, fetch only current solid layer.

Remaining build failure is in dagster-airflow, not related to this diff. Going to merge this.

Dec 5 2019, 11:03 PM
bengotow updated the diff for D1540: Break pipeline explorer and sidebar queries apart, fetch only current solid layer.

Fix additional TS issue not found locally

Dec 5 2019, 10:31 PM
bengotow updated the diff for D1540: Break pipeline explorer and sidebar queries apart, fetch only current solid layer.

Update data provided to our tests to match new root queries

Dec 5 2019, 10:00 PM
bengotow updated the diff for D1540: Break pipeline explorer and sidebar queries apart, fetch only current solid layer.

Update for linting

Dec 5 2019, 7:33 PM
bengotow added a comment to D1540: Break pipeline explorer and sidebar queries apart, fetch only current solid layer.

Will fix lint!

Dec 5 2019, 7:24 PM
bengotow planned changes to D1542: Considerably reduce volume of type data fetched by ConfigEditor.

Oh interesting! Will check this out this afternoon—I wonder if TypeScript didn't catch usage of some of this data somehow.

Dec 5 2019, 7:21 PM
bengotow added inline comments to D1542: Considerably reduce volume of type data fetched by ConfigEditor.
Dec 5 2019, 12:55 AM
bengotow created D1542: Considerably reduce volume of type data fetched by ConfigEditor.
Dec 5 2019, 12:54 AM