Page MenuHomePhabricator

bengotow (Ben Gotow)
User

Projects

User does not belong to any projects.

User Details

User Since
May 9 2019, 3:45 PM (15 w, 8 h)

Recent Activity

Yesterday

bengotow updated the diff for D869: Performance improvements based on fan-in-out example DAG.
  • Add prop name to shouldComponentUpdate
Thu, Aug 22, 9:51 PM
bengotow added inline comments to D869: Performance improvements based on fan-in-out example DAG.
Thu, Aug 22, 9:48 PM
bengotow added a comment to D875: Add test coverage of the pipeline to svg process using svg snapshots.

Yeah we definitely need to hide some of these files!

Thu, Aug 22, 9:30 PM
bengotow updated the diff for D875: Add test coverage of the pipeline to svg process using svg snapshots.
  • Add a test for a composite, add package cache so the tests pass
Thu, Aug 22, 9:02 PM
bengotow updated the diff for D875: Add test coverage of the pipeline to svg process using svg snapshots.
  • Add a test for a composite, add package cache so the tests pass
Thu, Aug 22, 9:01 PM
bengotow created D875: Add test coverage of the pipeline to svg process using svg snapshots.
Thu, Aug 22, 8:43 PM
bengotow added a comment to D844: Initial pass at custom modification of the Dagre layout for wide pipelines.

Hmm, this is /sorta/ incorporated into the snapshot tests, but I think we could definitely create some tests around dagre and this overall layout logic. Actually, we may be able to render out SVGs of given DAGs and then compare the SVGs, which would allow you to see WHAT changed when the code is modified / broken (I think otherwise this is just going to be comparing 200kb of JSON full of numbers and they'll just get updated blindly.) Will see if I can set this up today.

Thu, Aug 22, 6:54 PM
bengotow added a comment to D858: Overhauled run history view.

Thanks for the comments—will get these addressed tomorrow,

Thu, Aug 22, 6:52 AM
bengotow updated the diff for D869: Performance improvements based on fan-in-out example DAG.
  • Fix up comments
Thu, Aug 22, 6:40 AM
bengotow updated the summary of D869: Performance improvements based on fan-in-out example DAG.
Thu, Aug 22, 6:32 AM
bengotow updated the summary of D869: Performance improvements based on fan-in-out example DAG.
Thu, Aug 22, 6:29 AM
bengotow created D869: Performance improvements based on fan-in-out example DAG.
Thu, Aug 22, 6:22 AM

Wed, Aug 21

bengotow updated the summary of D844: Initial pass at custom modification of the Dagre layout for wide pipelines.
Wed, Aug 21, 7:08 PM
bengotow updated the diff for D858: Overhauled run history view.
  • Update snapshot tests
Wed, Aug 21, 6:59 PM
bengotow updated the summary of D858: Overhauled run history view.
Wed, Aug 21, 6:56 PM
bengotow created D858: Overhauled run history view.
Wed, Aug 21, 6:54 PM

Tue, Aug 20

bengotow accepted D846: Configurable fan-in/fan-out pipeline for testing rendering.
Tue, Aug 20, 6:19 PM
bengotow updated the diff for D844: Initial pass at custom modification of the Dagre layout for wide pipelines.

Rebase

Tue, Aug 20, 5:13 PM
bengotow updated the summary of D844: Initial pass at custom modification of the Dagre layout for wide pipelines.
Tue, Aug 20, 3:58 PM
bengotow created D844: Initial pass at custom modification of the Dagre layout for wide pipelines.
Tue, Aug 20, 3:56 PM

Thu, Aug 15

bengotow accepted D819: Add custom behavior to the CellTruncationProvider.

This looks great! This approach definitely seems more flexible.

Thu, Aug 15, 6:47 PM

Tue, Aug 13

bengotow created D805: Add dagster-dbt to install_dev_python_modules.
Tue, Aug 13, 2:33 PM
bengotow accepted D792: fix column resizing in run view.

Going to approve with comments, because I'm not certain that my suggested fix will work. If it doesn't, this seems like a good solution.

Tue, Aug 13, 2:21 PM
bengotow added inline comments to D792: fix column resizing in run view.
Tue, Aug 13, 2:20 PM

Mon, Aug 5

bengotow accepted D764: Fix typecast.
Mon, Aug 5, 9:34 PM
bengotow added inline comments to D719: Fix event handler.
Mon, Aug 5, 8:26 PM

Mon, Jul 29

bengotow updated the summary of D731: Assign fixed label hues to pyspark, spark, snowflake.
Mon, Jul 29, 6:58 PM
bengotow created D731: Assign fixed label hues to pyspark, spark, snowflake.
Mon, Jul 29, 6:57 PM
bengotow updated the summary of D730: Avoid repeated calls to readFragment to shorten new-logs critical path by 30%.
Mon, Jul 29, 6:48 PM
bengotow created D730: Avoid repeated calls to readFragment to shorten new-logs critical path by 30%.
Mon, Jul 29, 6:47 PM
bengotow updated the summary of D726: Separate "Loading..." and "Process starting..." status messages.
Mon, Jul 29, 5:54 PM
bengotow created D726: Separate "Loading..." and "Process starting..." status messages.
Mon, Jul 29, 5:54 PM
bengotow accepted D707: Lint.

Looks good! Thanks for making all these fixes.

Mon, Jul 29, 3:40 PM
bengotow accepted D721: Don't use a custom keyMapper.

Looks good to me! I think since we're blowing away and re-creating the cache now when the node set changes, we don't need the custom key mapping (you're correct that it was allowing you to filter the list without recomputing the height of the rows).

Mon, Jul 29, 3:38 PM

Thu, Jul 25

bengotow accepted D680: [dagit] lint adjustments.

Looks good to me! Definitely odd that the autogenerated classnames changed but that seems consistent with my experience 😅

Thu, Jul 25, 4:47 PM

Wed, Jul 24

bengotow added a reviewer for D690: Update client-side structured log presentation: Restricted Project.
Wed, Jul 24, 10:37 PM
bengotow created D690: Update client-side structured log presentation.
Wed, Jul 24, 10:36 PM

Jul 23 2019

bengotow accepted D674: [dagit] deps upgrade.

This looks great! Thanks for slogging through this. Made a few comments inline but nothing blocking.

Jul 23 2019, 10:13 PM
bengotow accepted D679: [dagit] fix types list error.

Looks good to me!

Jul 23 2019, 6:54 PM

Jul 18 2019

bengotow accepted D661: Remove unnecessary log messages.

👏 Glad to get these out of there!

Jul 18 2019, 7:44 PM
bengotow accepted D651: Migrate tslint to eslint.
Jul 18 2019, 7:42 PM
bengotow accepted D638: Configurable whitespace display in config editor.

Looks great! Nice to have this as an option!

Jul 18 2019, 7:36 PM
bengotow updated the diff for D527: Initial pass at structured log display in the run view.
  • Put input / output names in the label column
  • Use typecheck metadata rather than valueRepl (there appears to be none yet)
  • Improved metadata table layout
  • Fix truncation bug and placement of “read entire message” text
  • Displaying IO types in the structured logs, filtering by step type
  • Switch to using virtualized list instead of grid, since we do not want all grid features
  • Add draggable headers and switch to react-virtualized list component
  • Add timestamp heading, adjust styles and column names
  • Rebase
Jul 18 2019, 7:15 PM
bengotow added a comment to D527: Initial pass at structured log display in the run view.

Doesn't seem like the buildkite errors above are relevant to this diff, not sure what's going on with the python tests.

Jul 18 2019, 3:35 PM
bengotow added a comment to D527: Initial pass at structured log display in the run view.

Updated screenshot above!

Jul 18 2019, 3:12 PM
bengotow updated the diff for D527: Initial pass at structured log display in the run view.
  • Displaying IO types in the structured logs, filtering by step type
  • Switch to using virtualized list instead of grid, since we do not want all grid features
  • Add draggable headers and switch to react-virtualized list component
Jul 18 2019, 3:11 PM

Jul 17 2019

bengotow added a comment to D527: Initial pass at structured log display in the run view.

This should address 1-3 (table styling is more apparent and does not have headers, entire log view now has headers and the dividers can be dragged left/right to adjust column widths. sizing is saved between page loads. The input/output types are displayed in the standard blue) You can also now filter by node type by typing type:expectation or `type: step start, etc. (It just does a partial string match against the event graphQL typenames for now)

Jul 17 2019, 10:15 PM

Jul 15 2019

bengotow updated the diff for D527: Initial pass at structured log display in the run view.
  • Fix truncation bug and placement of “read entire message” text
  • Update failing snapshot tests
Jul 15 2019, 9:43 PM
bengotow added a comment to D527: Initial pass at structured log display in the run view.

Hey folks! Here's are a few screenshots of the latest changes. (Note the new table UI for the metadata entries)

Jul 15 2019, 9:40 PM
bengotow added reviewers for D527: Initial pass at structured log display in the run view: schrockn, alangenfeld.
Jul 15 2019, 8:05 PM
bengotow updated the diff for D527: Initial pass at structured log display in the run view.
  • Put input / output names in the label column
  • Use typecheck metadata rather than valueRepl (there appears to be none yet)
Jul 15 2019, 8:04 PM

Jul 3 2019

bengotow created D527: Initial pass at structured log display in the run view.
Jul 3 2019, 5:39 PM
bengotow added reviewers for D517: Disable the DEBUG log level by default: schrockn, alangenfeld.
Jul 3 2019, 3:20 AM
bengotow created D517: Disable the DEBUG log level by default.
Jul 3 2019, 3:20 AM

Jul 2 2019

bengotow updated the diff for D513: Add a Re-execute button to the pipeline run view.
  • Remove unused types file
Jul 2 2019, 8:41 PM
bengotow updated the diff for D513: Add a Re-execute button to the pipeline run view.
  • Use the “Repeat” icon for re-execution
Jul 2 2019, 8:39 PM
bengotow added reviewers for D513: Add a Re-execute button to the pipeline run view: schrockn, alangenfeld.
Jul 2 2019, 7:04 PM
bengotow created D513: Add a Re-execute button to the pipeline run view.
Jul 2 2019, 7:03 PM
bengotow updated the summary of D512: Pull refactoring/cleanup off experimental EP branch.
Jul 2 2019, 6:23 PM
bengotow created D512: Pull refactoring/cleanup off experimental EP branch.
Jul 2 2019, 6:22 PM

Jun 20 2019

bengotow added a comment to D442: yarn rm fs.

Looks good to me! Super weird this happened. (Also separately, it seems like calling a node module fs shouldn't be allowed...)

Jun 20 2019, 11:39 PM

Jun 19 2019

bengotow updated the summary of D427: Render composite mappings, external deps on the DAG.
Jun 19 2019, 3:56 AM
bengotow created D427: Render composite mappings, external deps on the DAG.
Jun 19 2019, 3:50 AM

Jun 17 2019

bengotow accepted D413: Events API.

This looks great! I like that this is generic and that we're also getting value type data from the backend (paths, json, etc.) so we don't have to do any guesswork when it comes to presenting the values. Good stuff!

Jun 17 2019, 3:45 PM

Jun 14 2019

bengotow updated the diff for D404: Initial improvements to the solid sidebar.
  • Fix padding when the pipeline is selected in the sidebar
  • Update the snapshot tests
Jun 14 2019, 8:52 PM
bengotow updated the summary of D404: Initial improvements to the solid sidebar.
Jun 14 2019, 8:30 PM
bengotow updated the summary of D404: Initial improvements to the solid sidebar.
Jun 14 2019, 8:30 PM
bengotow created D404: Initial improvements to the solid sidebar.
Jun 14 2019, 8:25 PM

Jun 9 2019

bengotow accepted D347: diff for Ben.

Looks good to me! I guess we may want to think about whether handleID is still the best word for this. Implementation aside, it seems like solids could have a unique key or id, or if we want to formalize the dot-notation used for them, we could call it a path.

Jun 9 2019, 10:10 PM

Jun 5 2019

bengotow accepted D301: Do not pin to python37 executable in append_generated.

Looks good to me! One minor nit is that a few months ago generate-types started to require that it is run inside the dagster venv. I wonder if there's a super easy way to bail / alert the user if they run generate-types outside the env that will run the python script happily? Right now it blows up in a bit of a cryptic way:

Jun 5 2019, 4:40 PM

Jun 3 2019

bengotow updated the diff for D205: Add support for composite solids to the Explore tab.
  • Remove unneeded file modified every time generate-types runs
  • Generate types using Python 3.7 to match BuildKite behavior
Jun 3 2019, 7:22 PM
bengotow updated the diff for D205: Add support for composite solids to the Explore tab.
  • Mark that @types/color is a devDependency
  • Switch from indexOf + length to split+pop
  • Add ability to double click out of a composite solid by clicking the dark gray background
  • Finally have a good solution for animation—CSS rect transition + react keys
  • Use the solid tags area to display an “Expand” button
  • Slow down the animation a bit
  • Click solid before expanding it to prepare the initial animation state
  • Move the label on the parent solid container inside to prevent rect bounds from shifting when you zoom
  • Looks like BuildKite wants types rebuilt
  • Remove unneeded file modified every time generate-types runs
Jun 3 2019, 4:03 PM
bengotow updated the diff for D205: Add support for composite solids to the Explore tab.
  • Remove unneeded file modified every time generate-types runs
Jun 3 2019, 3:19 PM

Jun 1 2019

bengotow updated the diff for D205: Add support for composite solids to the Explore tab.
  • Looks like BuildKite wants types rebuilt
Jun 1 2019, 4:49 PM

May 31 2019

bengotow updated the diff for D205: Add support for composite solids to the Explore tab.
  • Slow down the animation a bit
  • Click solid before expanding it to prepare the initial animation state
  • Move the label on the parent solid container inside to prevent rect bounds from shifting when you zoom
May 31 2019, 7:10 PM
bengotow updated the diff for D205: Add support for composite solids to the Explore tab.
  • Switch from indexOf + length to split+pop
  • Add ability to double click out of a composite solid by clicking the dark gray background
  • Finally have a good solution for animation—CSS rect transition + react keys
  • Use the solid tags area to display an “Expand” button
May 31 2019, 7:03 PM

May 28 2019

bengotow accepted D201: EnvironmentSchema in GraphQL.
May 28 2019, 8:39 PM
bengotow added a comment to D201: EnvironmentSchema in GraphQL.

This looks great. I think making this a top level GraphQL query that requires the execution selector makes it more clear that the environment is dependent on the solid subset. I think this is also a nice step toward lowering the systems dependence on the idea of a pipeline. (In case we want to allow any composite solid to be an addressable unit of execution one day, etc.)

May 28 2019, 8:38 PM
bengotow added a comment to D205: Add support for composite solids to the Explore tab.

Hey folks! Sounds good—we are very far from being able to animate this transition, but I think we can hack something together that communicates what you're looking for. In terms of the zooming in I agree it's a lot of clicks, but I think it's worth considering how often the user will actually want to look inside the solids. I figured this was something you'd do infrequently, and clicking around and having it "jump" inside what you were looking at would actually be more annoying. Regardless, should be easy to come up with something else. There's also an "Expand" button in the sidebar, and we could just put an explicit button in the DAG view as well.

May 28 2019, 3:25 PM

May 24 2019

bengotow updated the diff for D205: Add support for composite solids to the Explore tab.
  • Mark that @types/color is a devDependency
May 24 2019, 10:04 PM
bengotow updated the summary of D205: Add support for composite solids to the Explore tab.
May 24 2019, 10:00 PM
bengotow created D205: Add support for composite solids to the Explore tab.
May 24 2019, 9:51 PM

May 21 2019

bengotow accepted D171: Improve display of materializations and expectations.

This looks great to me! Nice visual improvement. It's definitely less generic if we break down "display events" into materializations + expectations and pass those two categories in explicitly, but that does allow for a more customized presentation of each type. I think an alternative would be to allow for display events to have children and do the disclosure triangles in a super generic way, but given that these are probably the two major categories it seems fine to just handle each one.

May 21 2019, 11:58 PM

May 20 2019

bengotow accepted D168: [dagster] fix SolidHandle schema.

Looks great! Simple enough 👍

May 20 2019, 6:03 PM

May 18 2019

bengotow committed R1:1d30977b7ffa: Initial pass at explore tab changes + SVG rendering of mappings (authored by bengotow).
Initial pass at explore tab changes + SVG rendering of mappings
May 18 2019, 1:50 AM

May 17 2019

bengotow accepted D153: fix react key warning in SidebarSolidInfo.

Looks good to me!

May 17 2019, 8:14 PM

May 9 2019

GitHub <noreply@github.com> committed R1:2294add60c5c: Short term improvements to the execution plan view (#1332) (authored by bengotow).
Short term improvements to the execution plan view (#1332)
May 9 2019, 4:42 PM
GitHub <noreply@github.com> committed R1:97821d764e3a: Merge c09bd4cd9021c645141c760b78bcb7e7f0736dd9 into… (authored by bengotow).
Merge c09bd4cd9021c645141c760b78bcb7e7f0736dd9 into…
May 9 2019, 4:32 PM
bengotow committed R1:c09bd4cd9021: Move icons out to external SVGs (authored by bengotow).
Move icons out to external SVGs
May 9 2019, 4:32 PM
bengotow committed R1:df4b7ecc4a75: Switcch to a new display event interface which includes subitems (authored by bengotow).
Switcch to a new display event interface which includes subitems
May 9 2019, 4:32 PM
bengotow committed R1:9127e4d4d637: Make intermediate/materialization support more generic client-side (authored by bengotow).
Make intermediate/materialization support more generic client-side
May 9 2019, 4:32 PM
bengotow committed R1:947920aa64b5: Remove double circle around disabled re-execute button (authored by bengotow).
Remove double circle around disabled re-execute button
May 9 2019, 4:32 PM
GitHub <noreply@github.com> committed R1:e9175f98ba07: Merge 804793742867f7f84ed13defb2a4533bcd99cae8 into… (authored by bengotow).
Merge 804793742867f7f84ed13defb2a4533bcd99cae8 into…
May 9 2019, 4:14 PM