Page MenuHomePhabricator
Feed Advanced Search

Wed, Sep 18

bengotow updated the diff for D980: Move log-dervied stats to Python, expose via graphql as `run.stats`.

Ugh apparently I have to rebase and not merge master

Wed, Sep 18, 7:26 PM
bengotow updated the diff for D980: Move log-dervied stats to Python, expose via graphql as `run.stats`.
  • Merge branch 'master' into bengotow/-run-stats
Wed, Sep 18, 7:23 PM
bengotow added inline comments to D1057: Add a floating context panel to the Dagit config editor.
Wed, Sep 18, 7:17 PM
bengotow updated the summary of D1057: Add a floating context panel to the Dagit config editor.
Wed, Sep 18, 7:15 PM
bengotow created D1057: Add a floating context panel to the Dagit config editor.
Wed, Sep 18, 7:13 PM
bengotow updated the diff for D1044: Add re-execute to the pipeline runs dropdown menus.
  • Remove cancel
Wed, Sep 18, 7:09 PM
bengotow updated the diff for D980: Move log-dervied stats to Python, expose via graphql as `run.stats`.
  • Avoid Date.now, add better explanation of sort logic
  • Show number of steps out of total
  • Run check in PipelineRunStats constructor
Wed, Sep 18, 7:04 PM

Tue, Sep 17

bengotow updated the diff for D1044: Add re-execute to the pipeline runs dropdown menus.

Update the text labels after discussion with team

Tue, Sep 17, 6:24 PM
bengotow created D1044: Add re-execute to the pipeline runs dropdown menus.
Tue, Sep 17, 2:29 PM

Mon, Sep 16

bengotow updated the diff for D1029: Replace runs search bar with a nice autocompleting tokenized field.

Rebuild snapshot tests

Mon, Sep 16, 4:52 PM
bengotow added a comment to D1029: Replace runs search bar with a nice autocompleting tokenized field.

Good catch—I switched this over to using onFocus and onBlur on the input to manage the open / close state rather than inheriting behavior from the popover and that seems to work much better!

Mon, Sep 16, 4:51 PM
bengotow updated the diff for D1029: Replace runs search bar with a nice autocompleting tokenized field.

Rebase and address feedback

Mon, Sep 16, 4:50 PM

Fri, Sep 13

bengotow created D1029: Replace runs search bar with a nice autocompleting tokenized field.
Fri, Sep 13, 4:15 PM
bengotow updated the diff for D985: Basic implementation of search through run history.

Rebasing

Fri, Sep 13, 4:04 PM
bengotow updated the diff for D985: Basic implementation of search through run history.

Make runId searchable, support /runs?q=<term> so nobody is copy-pasting runIds

Fri, Sep 13, 2:58 PM
bengotow added inline comments to D980: Move log-dervied stats to Python, expose via graphql as `run.stats`.
Fri, Sep 13, 2:25 PM
bengotow added a comment to D985: Basic implementation of search through run history.

Sure! Don't see why not. While I'm at it I'll make it so this search field can be populated from the query string - If there's some other place people are copy-pasting runIds from, it'd be cool to switch to a dagit link.

Fri, Sep 13, 3:51 AM
bengotow planned changes to D980: Move log-dervied stats to Python, expose via graphql as `run.stats`.

Hmm, I'll see if we can pull the total number of steps - being able to show a total count rather than accumulating them as we go would be nice. I think we may be pulling the execution plan into the UI already so it's probably not a big leap.

Fri, Sep 13, 3:49 AM
bengotow abandoned D943: Mock more of the GraphQL queries, add snapshot test for new runs page.

Sounds good! I think this snapshot testing isn't working out very well because the mock data for Dagit is very hard to generate / very large compared to your typical react application, and is still changing often enough that we need to be able to re-generate it or it'll fall out of sync with the actual runtime values pretty quickly. I'm going to abandon this and I'll see if we can start writing more component-level / panel-level tests that might require less sophisticated / more piece-meal mock data.

Fri, Sep 13, 3:46 AM

Mon, Sep 9

bengotow created D985: Basic implementation of search through run history.
Mon, Sep 9, 10:32 PM
bengotow updated the diff for D980: Move log-dervied stats to Python, expose via graphql as `run.stats`.
  • Fix sort of runs with no start time
Mon, Sep 9, 10:11 PM
bengotow updated the summary of D980: Move log-dervied stats to Python, expose via graphql as `run.stats`.
Mon, Sep 9, 4:45 PM
bengotow created D980: Move log-dervied stats to Python, expose via graphql as `run.stats`.
Mon, Sep 9, 4:43 PM

Thu, Sep 5

bengotow updated the diff for D945: Add autocompletion of bool true/false values in config editor.

Switch to True/False

Thu, Sep 5, 6:11 PM
bengotow added inline comments to D945: Add autocompletion of bool true/false values in config editor.
Thu, Sep 5, 6:06 PM
bengotow updated the summary of D945: Add autocompletion of bool true/false values in config editor.
Thu, Sep 5, 4:58 PM
bengotow created D945: Add autocompletion of bool true/false values in config editor.
Thu, Sep 5, 4:57 PM
bengotow created D944: Fix the example use of `logdir` to be `log-dir`.
Thu, Sep 5, 3:26 PM
bengotow created D943: Mock more of the GraphQL queries, add snapshot test for new runs page.
Thu, Sep 5, 3:24 PM
bengotow added inline comments to D855: Create Scheduler UI in Dagit.
Thu, Sep 5, 2:52 PM

Tue, Sep 3

bengotow added a comment to D920: Fix hook issues, stop serializing modal contents to JSON.

Sounds good! The reason this exists vs. doing each dialog inline is just because the dialog doesn't animate open / closed unless it's container is in the DOM when it's open AND closed (set via the <Dialog isOpen={}> in this case). So if you're not careful, you can end up with scenarios where every single row of a table contains an invisible not-yet-opened dialog DOM tree (or you just lose the animation, which would be better).

Tue, Sep 3, 9:15 PM
bengotow added inline comments to D920: Fix hook issues, stop serializing modal contents to JSON.
Tue, Sep 3, 6:30 PM
bengotow created D920: Fix hook issues, stop serializing modal contents to JSON.
Tue, Sep 3, 6:26 PM
bengotow updated the diff for D905: Upgrade apollo, switch to hooks where it increases readability.
  • Turn on React hooks eslint rules
Tue, Sep 3, 5:39 PM
bengotow added a comment to D905: Upgrade apollo, switch to hooks where it increases readability.

Ahh good call re: React hooks eslint rules. I turned them on and verified that they work by breaking hooks rules in a few places. Very slick. Will wait for this build to pass and get this merged!

Tue, Sep 3, 5:38 PM
bengotow accepted D909: force run view under pipeline in url namespace.

Just a few inline comments—nothing blocking!

Tue, Sep 3, 5:30 PM

Thu, Aug 29

bengotow updated the diff for D905: Upgrade apollo, switch to hooks where it increases readability.

Rebase

Thu, Aug 29, 12:15 AM
bengotow created D905: Upgrade apollo, switch to hooks where it increases readability.
Thu, Aug 29, 12:09 AM

Wed, Aug 28

bengotow accepted D904: Fix black.
Wed, Aug 28, 11:11 PM
bengotow updated the diff for D858: Overhauled run history view.
  • Update autogenerated types
Wed, Aug 28, 11:01 PM
bengotow updated the diff for D858: Overhauled run history view.
  • Update app snapshot tests to reflect new page URLs
Wed, Aug 28, 10:59 PM
bengotow updated the diff for D858: Overhauled run history view.
  • Update snapshot tests
  • Remove complexity from CustomAlertProvider in favor of passing a ReactNode
  • Change paths to /explore/<pipeline> instead of <pipeline>/explore
  • Remove occurrences of if \([^\)]*\)$ in the TS (two line no-bracket ifs)
  • Consolidat time and stepKey formatting to the same utils
  • Tick up the elapsed time when runs are in-flight
  • Rebase
  • Switch <StorageProvider> to a hook to clean up the redirect
  • Move the hack to a separate page to keep things really nice and clean
  • Move the hack out of the main pipeline execution page
  • Revert changes to apollo version
  • Fix item missing in package cache
  • Rebase on master
Wed, Aug 28, 10:55 PM
bengotow updated the diff for D858: Overhauled run history view.
  • Fix item missing in package cache
Wed, Aug 28, 10:41 PM
bengotow closed D875: Add test coverage of the pipeline to svg process using svg snapshots.

Due to some arc land mishaps this merged in 1634aa48368675355515d04166ed26d0ead7e9f1

Wed, Aug 28, 10:32 PM
bengotow closed D902: Use dagster-graphql for mock response generation.

Due to some arc land mishaps this merged in 1634aa48368675355515d04166ed26d0ead7e9f1

Wed, Aug 28, 10:32 PM
bengotow updated the diff for D902: Use dagster-graphql for mock response generation.
  • Revert use of jest for download-schema
Wed, Aug 28, 10:17 PM
bengotow added inline comments to D902: Use dagster-graphql for mock response generation.
Wed, Aug 28, 10:16 PM
bengotow updated the diff for D858: Overhauled run history view.
  • Revert changes to apollo version
Wed, Aug 28, 10:15 PM
bengotow updated the diff for D858: Overhauled run history view.
  • Move the hack out of the main pipeline execution page
Wed, Aug 28, 9:36 PM
bengotow updated the diff for D858: Overhauled run history view.
  • Switch <StorageProvider> to a hook to clean up the redirect
  • Move the hack to a separate page to keep things really nice and clean
Wed, Aug 28, 9:32 PM
bengotow created D902: Use dagster-graphql for mock response generation.
Wed, Aug 28, 7:58 PM

Tue, Aug 27

bengotow updated the diff for D875: Add test coverage of the pipeline to svg process using svg snapshots.

Fixing base

Tue, Aug 27, 6:08 PM
bengotow updated the diff for D875: Add test coverage of the pipeline to svg process using svg snapshots.
  • Remove accidental notes doc
Tue, Aug 27, 6:06 PM
bengotow added a comment to D875: Add test coverage of the pipeline to svg process using svg snapshots.

ahh good call! Will get this switched over to use the dagster-graphql CLI, didn't know about that 👍

Tue, Aug 27, 6:03 PM
bengotow added a comment to D858: Overhauled run history view.

Hey @alangenfeld - this should be ready for another go. I rebased on master today but I wasn't able to reproduce the problem you saw with all the historical runs showing gray dots. Any idea what might have caused that?

Tue, Aug 27, 6:01 PM
bengotow updated the diff for D858: Overhauled run history view.
  • Update snapshot tests
  • Remove complexity from CustomAlertProvider in favor of passing a ReactNode
  • Change paths to /explore/<pipeline> instead of <pipeline>/explore
  • Remove occurrences of if \([^\)]*\)$ in the TS (two line no-bracket ifs)
  • Consolidat time and stepKey formatting to the same utils
  • Tick up the elapsed time when runs are in-flight
  • Rebase
Tue, Aug 27, 5:59 PM
bengotow added a comment to D875: Add test coverage of the pipeline to svg process using svg snapshots.

Oh nice! Didn't realize that was a convention - renamed this and we should be good to go.

Tue, Aug 27, 4:07 PM
bengotow updated the diff for D875: Add test coverage of the pipeline to svg process using svg snapshots.
  • Use path.join for windows-friendly paths
Tue, Aug 27, 4:07 PM
bengotow updated the diff for D875: Add test coverage of the pipeline to svg process using svg snapshots.
  • Move data / snapshots to dunder’d dirs
Tue, Aug 27, 4:01 PM

Aug 22 2019

bengotow updated the diff for D869: Performance improvements based on fan-in-out example DAG.
  • Add prop name to shouldComponentUpdate
Aug 22 2019, 9:51 PM
bengotow added inline comments to D869: Performance improvements based on fan-in-out example DAG.
Aug 22 2019, 9:48 PM
bengotow added a comment to D875: Add test coverage of the pipeline to svg process using svg snapshots.

Yeah we definitely need to hide some of these files!

Aug 22 2019, 9:30 PM
bengotow updated the diff for D875: Add test coverage of the pipeline to svg process using svg snapshots.
  • Add a test for a composite, add package cache so the tests pass
Aug 22 2019, 9:02 PM
bengotow updated the diff for D875: Add test coverage of the pipeline to svg process using svg snapshots.
  • Add a test for a composite, add package cache so the tests pass
Aug 22 2019, 9:01 PM
bengotow created D875: Add test coverage of the pipeline to svg process using svg snapshots.
Aug 22 2019, 8:43 PM
bengotow added a comment to D844: Initial pass at custom modification of the Dagre layout for wide pipelines.

Hmm, this is /sorta/ incorporated into the snapshot tests, but I think we could definitely create some tests around dagre and this overall layout logic. Actually, we may be able to render out SVGs of given DAGs and then compare the SVGs, which would allow you to see WHAT changed when the code is modified / broken (I think otherwise this is just going to be comparing 200kb of JSON full of numbers and they'll just get updated blindly.) Will see if I can set this up today.

Aug 22 2019, 6:54 PM
bengotow added a comment to D858: Overhauled run history view.

Thanks for the comments—will get these addressed tomorrow,

Aug 22 2019, 6:52 AM
bengotow updated the diff for D869: Performance improvements based on fan-in-out example DAG.
  • Fix up comments
Aug 22 2019, 6:40 AM
bengotow updated the summary of D869: Performance improvements based on fan-in-out example DAG.
Aug 22 2019, 6:32 AM
bengotow updated the summary of D869: Performance improvements based on fan-in-out example DAG.
Aug 22 2019, 6:29 AM
bengotow created D869: Performance improvements based on fan-in-out example DAG.
Aug 22 2019, 6:22 AM

Aug 21 2019

bengotow updated the summary of D844: Initial pass at custom modification of the Dagre layout for wide pipelines.
Aug 21 2019, 7:08 PM
bengotow updated the diff for D858: Overhauled run history view.
  • Update snapshot tests
Aug 21 2019, 6:59 PM
bengotow updated the summary of D858: Overhauled run history view.
Aug 21 2019, 6:56 PM
bengotow created D858: Overhauled run history view.
Aug 21 2019, 6:54 PM

Aug 20 2019

bengotow accepted D846: Configurable fan-in/fan-out pipeline for testing rendering.
Aug 20 2019, 6:19 PM
bengotow updated the diff for D844: Initial pass at custom modification of the Dagre layout for wide pipelines.

Rebase

Aug 20 2019, 5:13 PM
bengotow updated the summary of D844: Initial pass at custom modification of the Dagre layout for wide pipelines.
Aug 20 2019, 3:58 PM
bengotow created D844: Initial pass at custom modification of the Dagre layout for wide pipelines.
Aug 20 2019, 3:56 PM

Aug 15 2019

bengotow accepted D819: Add custom behavior to the CellTruncationProvider.

This looks great! This approach definitely seems more flexible.

Aug 15 2019, 6:47 PM

Aug 13 2019

bengotow created D805: Add dagster-dbt to install_dev_python_modules.
Aug 13 2019, 2:33 PM
bengotow accepted D792: fix column resizing in run view.

Going to approve with comments, because I'm not certain that my suggested fix will work. If it doesn't, this seems like a good solution.

Aug 13 2019, 2:21 PM
bengotow added inline comments to D792: fix column resizing in run view.
Aug 13 2019, 2:20 PM

Aug 5 2019

bengotow accepted D764: Fix typecast.
Aug 5 2019, 9:34 PM
bengotow added inline comments to D719: Fix event handler.
Aug 5 2019, 8:26 PM

Jul 29 2019

bengotow updated the summary of D731: Assign fixed label hues to pyspark, spark, snowflake.
Jul 29 2019, 6:58 PM
bengotow created D731: Assign fixed label hues to pyspark, spark, snowflake.
Jul 29 2019, 6:57 PM
bengotow updated the summary of D730: Avoid repeated calls to readFragment to shorten new-logs critical path by 30%.
Jul 29 2019, 6:48 PM
bengotow created D730: Avoid repeated calls to readFragment to shorten new-logs critical path by 30%.
Jul 29 2019, 6:47 PM
bengotow updated the summary of D726: Separate "Loading..." and "Process starting..." status messages.
Jul 29 2019, 5:54 PM
bengotow created D726: Separate "Loading..." and "Process starting..." status messages.
Jul 29 2019, 5:54 PM
bengotow accepted D707: Lint.

Looks good! Thanks for making all these fixes.

Jul 29 2019, 3:40 PM
bengotow accepted D721: Don't use a custom keyMapper.

Looks good to me! I think since we're blowing away and re-creating the cache now when the node set changes, we don't need the custom key mapping (you're correct that it was allowing you to filter the list without recomputing the height of the rows).

Jul 29 2019, 3:38 PM

Jul 25 2019

bengotow accepted D680: [dagit] lint adjustments.

Looks good to me! Definitely odd that the autogenerated classnames changed but that seems consistent with my experience 😅

Jul 25 2019, 4:47 PM

Jul 24 2019

bengotow added a reviewer for D690: Update client-side structured log presentation: Restricted Project.
Jul 24 2019, 10:37 PM
bengotow created D690: Update client-side structured log presentation.
Jul 24 2019, 10:36 PM

Jul 23 2019

bengotow accepted D674: [dagit] deps upgrade.

This looks great! Thanks for slogging through this. Made a few comments inline but nothing blocking.

Jul 23 2019, 10:13 PM
bengotow accepted D679: [dagit] fix types list error.

Looks good to me!

Jul 23 2019, 6:54 PM

Jul 18 2019

bengotow accepted D661: Remove unnecessary log messages.

👏 Glad to get these out of there!

Jul 18 2019, 7:44 PM
bengotow accepted D651: Migrate tslint to eslint.
Jul 18 2019, 7:42 PM