Page MenuHomeElementl

alangenfeld (Alex Langenfeld)
UserAdministrator

Projects

User does not belong to any projects.

User Details

User Since
Mar 20 2019, 8:25 PM (108 w, 4 d)
Roles
Administrator

Recent Activity

Fri, Apr 16

alangenfeld accepted D7491: include stack trace in repo failure.

olmec

Fri, Apr 16, 10:29 PM
alangenfeld published D7491: include stack trace in repo failure for review.
Fri, Apr 16, 10:28 PM
alangenfeld added inline comments to D7295: raise exceptions from grpc methods.
Fri, Apr 16, 10:23 PM
alangenfeld added inline comments to D7295: raise exceptions from grpc methods.
Fri, Apr 16, 9:47 PM
alangenfeld requested review of D7489: refactor solid input validation.
Fri, Apr 16, 9:35 PM
alangenfeld accepted D7486: change test to customize timeout to minimize test flakes.
Fri, Apr 16, 8:35 PM
alangenfeld accepted D7419: Improved error message when missing config.

i think this is a clear step forward even if its json sample in yaml land dagit

Fri, Apr 16, 6:10 PM
alangenfeld accepted D7483: fix compute log subscriptions.

golem

Fri, Apr 16, 6:05 PM
alangenfeld added a comment to D7446: default config schema to Any.

should any docs or tutorials get updated in this diff as well?

Fri, Apr 16, 4:28 PM
alangenfeld accepted D7459: Use cleaner NamedTuple syntax for InputContext and OutputContext.

sweet thanks much

Fri, Apr 16, 4:26 PM
alangenfeld closed D7468: [mypy] pipeline & graph definitions.
Fri, Apr 16, 2:09 PM
alangenfeld committed R1:dc61de9defb4: [mypy] pipeline & graph definitions (authored by alangenfeld).
[mypy] pipeline & graph definitions
Fri, Apr 16, 2:09 PM

Thu, Apr 15

alangenfeld added inline comments to D7261: [RFC] change DagsterInstance.get behavior.
Thu, Apr 15, 9:45 PM
alangenfeld updated the diff for D7261: [RFC] change DagsterInstance.get behavior.

copy refresh - support dagster pipeline execute

Thu, Apr 15, 9:42 PM
alangenfeld added a comment to D7459: Use cleaner NamedTuple syntax for InputContext and OutputContext.

oh wait - whats the testing story for IO managers? Do we expect users to create these by hand in test scenarios ? We might need to the typing.NamedTuple with __new__ version

Thu, Apr 15, 9:40 PM
alangenfeld requested review of D7468: [mypy] pipeline & graph definitions.
Thu, Apr 15, 9:29 PM
alangenfeld requested changes to D7463: [mypy] Events mypy.

switch the NamedTuples to still have __new__ but be typed

Thu, Apr 15, 9:02 PM
alangenfeld accepted D7464: [Depends on D7374] solid definition mypy.
Thu, Apr 15, 8:48 PM
alangenfeld resigned from D7462: Limit the scope of hook context.

want @yuhan to sign off on this one

Thu, Apr 15, 8:46 PM
alangenfeld requested changes to D7459: Use cleaner NamedTuple syntax for InputContext and OutputContext.

being annoying but i think its worth

Thu, Apr 15, 8:40 PM
alangenfeld resigned from D7438: Revamp the configuration concept section.
Thu, Apr 15, 6:08 PM
alangenfeld accepted D7436: Add --short flag to dagster-release version (for automation).

whynot

Thu, Apr 15, 6:00 PM
alangenfeld added a comment to D7374: execution context naming revamp, remove resources and intermediate storage from run worker.

land late tonight / early tomorrow for max soak between now and next release

Thu, Apr 15, 5:54 PM
alangenfeld accepted D7374: execution context naming revamp, remove resources and intermediate storage from run worker.

😗 👌

Thu, Apr 15, 5:53 PM
alangenfeld accepted D7445: some type annotations for config-related code.

weedle

Thu, Apr 15, 2:33 PM

Wed, Apr 14

alangenfeld closed D7376: make extra resources config optional.
Wed, Apr 14, 10:02 PM
alangenfeld committed R1:89baa28080f5: make extra resources config optional (authored by alangenfeld).
make extra resources config optional
Wed, Apr 14, 10:02 PM
alangenfeld updated the diff for D7376: make extra resources config optional.

typo

Wed, Apr 14, 5:45 PM
alangenfeld updated the diff for D7376: make extra resources config optional.

rebase, do some refactoring and leave useful comments explaning current setup

Wed, Apr 14, 5:44 PM
alangenfeld added inline comments to D7374: execution context naming revamp, remove resources and intermediate storage from run worker.
Wed, Apr 14, 4:48 PM
alangenfeld added a reviewer for D7376: make extra resources config optional: prha.
Wed, Apr 14, 4:14 PM
alangenfeld closed D7406: [dagstermill] support Failure and RetryRequested.
Wed, Apr 14, 4:08 PM
alangenfeld committed R1:d6502d922d66: [dagstermill] support Failure and RetryRequested (authored by alangenfeld).
[dagstermill] support Failure and RetryRequested
Wed, Apr 14, 4:08 PM
alangenfeld closed D7412: dont re-raise errors caught by pipeline failure.
Wed, Apr 14, 3:37 PM
alangenfeld committed R1:1477bd1ec02b: dont re-raise errors caught by pipeline failure (authored by alangenfeld).
dont re-raise errors caught by pipeline failure
Wed, Apr 14, 3:37 PM

Tue, Apr 13

alangenfeld accepted D7416: only run example tests on py3.8.
Tue, Apr 13, 10:04 PM
alangenfeld updated the diff for D7412: dont re-raise errors caught by pipeline failure.

lint

Tue, Apr 13, 9:42 PM
alangenfeld requested review of D7412: dont re-raise errors caught by pipeline failure.

just lint issues which i will fix

Tue, Apr 13, 9:41 PM
alangenfeld closed D7414: fix dagit.
Tue, Apr 13, 8:58 PM
alangenfeld committed R1:0643cdb1d1eb: fix dagit (authored by alangenfeld).
fix dagit
Tue, Apr 13, 8:58 PM
alangenfeld added a comment to D7406: [dagstermill] support Failure and RetryRequested.

To be clear in vanilla solids you still have to raise these right?

Tue, Apr 13, 7:05 PM
alangenfeld requested review of D7406: [dagstermill] support Failure and RetryRequested.
Tue, Apr 13, 4:53 PM

Mon, Apr 12

alangenfeld updated the summary of D7261: [RFC] change DagsterInstance.get behavior.
Mon, Apr 12, 10:05 PM
alangenfeld added inline comments to D7374: execution context naming revamp, remove resources and intermediate storage from run worker.
Mon, Apr 12, 10:01 PM
alangenfeld added a reviewer for D7261: [RFC] change DagsterInstance.get behavior: sandyryza.
Mon, Apr 12, 9:56 PM
alangenfeld requested changes to D7374: execution context naming revamp, remove resources and intermediate storage from run worker.
  • add comments / docblocks to the context classes explaining the thought in the current structure
  • make sure to turn on integration tests too
Mon, Apr 12, 9:54 PM
alangenfeld added inline comments to D7374: execution context naming revamp, remove resources and intermediate storage from run worker.
Mon, Apr 12, 9:48 PM
alangenfeld closed D7381: [RFC] rm all legacy_examples.
Mon, Apr 12, 9:32 PM
alangenfeld committed R1:acc93ecdb532: [RFC] rm all legacy_examples (authored by alangenfeld).
[RFC] rm all legacy_examples
Mon, Apr 12, 9:31 PM
alangenfeld added inline comments to D7376: make extra resources config optional.
Mon, Apr 12, 6:14 PM
alangenfeld closed D7388: pin jupyter-client.
Mon, Apr 12, 4:04 PM
alangenfeld committed R1:64025b728e9c: pin jupyter-client (authored by alangenfeld).
pin jupyter-client
Mon, Apr 12, 4:04 PM

Fri, Apr 9

alangenfeld requested review of D7381: [RFC] rm all legacy_examples.
Fri, Apr 9, 10:54 PM
alangenfeld requested review of D7376: make extra resources config optional.
Fri, Apr 9, 10:33 PM
alangenfeld added a comment to D7374: execution context naming revamp, remove resources and intermediate storage from run worker.

definitely loving this direction - I think we can trim even further as highlighted with some inline comments

Fri, Apr 9, 9:12 PM
alangenfeld added a comment to D7374: execution context naming revamp, remove resources and intermediate storage from run worker.

definitely worth adding comments to all the classes

Fri, Apr 9, 7:52 PM
alangenfeld accepted D7365: fix partition run matrix to handle composite solids / dynamic solids.

seems reasonable to me

Fri, Apr 9, 4:07 PM
alangenfeld requested changes to D7364: Dump dagster debug exports in integration tests.

to your queue

Fri, Apr 9, 2:45 PM
alangenfeld added a comment to D7364: Dump dagster debug exports in integration tests.

I think it should be possible to update the test callsites to make sure a snapshot is recorded - likely to be a net improvement to the system depending on where you can enable it.

Fri, Apr 9, 2:45 PM
alangenfeld accepted D7300: Move user-code-dependent executor validation to execution plan generation, rather than the run worker.

coolbeans

Fri, Apr 9, 2:34 PM
alangenfeld closed D7347: fix optional solid config for input config.
Fri, Apr 9, 2:29 PM
alangenfeld committed R1:51c00d8e0431: fix optional solid config for input config (authored by alangenfeld).
fix optional solid config for input config
Fri, Apr 9, 2:29 PM

Thu, Apr 8

alangenfeld added inline comments to D7347: fix optional solid config for input config.
Thu, Apr 8, 10:17 PM
alangenfeld updated the diff for D7347: fix optional solid config for input config.

more tests and comments

Thu, Apr 8, 10:10 PM
alangenfeld planned changes to D7347: fix optional solid config for input config.

ya let me take another pass at this

Thu, Apr 8, 9:47 PM
alangenfeld updated the summary of D7347: fix optional solid config for input config.
Thu, Apr 8, 9:45 PM
alangenfeld updated the summary of D7347: fix optional solid config for input config.
Thu, Apr 8, 9:39 PM
alangenfeld updated the summary of D7347: fix optional solid config for input config.
Thu, Apr 8, 8:59 PM
alangenfeld accepted D7351: [1/2] Fix re-execution if execution plan is delegated.
Thu, Apr 8, 8:26 PM
alangenfeld added inline comments to D7351: [1/2] Fix re-execution if execution plan is delegated.
Thu, Apr 8, 7:49 PM
alangenfeld requested changes to D7348: [RFC] Add all executable steps to known execution state.

+ the system.py changes

Thu, Apr 8, 5:24 PM
alangenfeld added a comment to D7337: RFC solid-hook context.failure_event.

quick drive by thoughts

  • since hooks happen in process I think we should have the actual exception object available on failure
  • I think we should probably have all the events from the step available, then maybe utilities for finding the success/failure hook from the full list
Thu, Apr 8, 3:12 PM

Wed, Apr 7

alangenfeld requested review of D7347: fix optional solid config for input config.
Wed, Apr 7, 10:29 PM
alangenfeld closed D7330: better error message for @resource param issues.
Wed, Apr 7, 10:09 PM
alangenfeld committed R1:d5c44bfba931: better error message for @resource param issues (authored by alangenfeld).
better error message for @resource param issues
Wed, Apr 7, 10:09 PM
alangenfeld added inline comments to D7261: [RFC] change DagsterInstance.get behavior.
Wed, Apr 7, 9:35 PM
alangenfeld added a comment to D7300: Move user-code-dependent executor validation to execution plan generation, rather than the run worker.

ExecutorProcessSetting giving you pause?

Wed, Apr 7, 9:30 PM
alangenfeld accepted D7346: Fix integration tests.
Wed, Apr 7, 9:01 PM
alangenfeld closed D7342: [mypy] environment_configs.py.
Wed, Apr 7, 5:59 PM
alangenfeld committed R1:adb029ebf1d8: [mypy] environment_configs.py (authored by alangenfeld).
[mypy] environment_configs.py
Wed, Apr 7, 5:59 PM
alangenfeld added a comment to D7342: [mypy] environment_configs.py.

mypy only does static type checking, so we still need runtime checking. We still want to use check for validating values as they enter the program from external sources, like what users pass in or what we deserialize out of storage or from cross process communication.

Wed, Apr 7, 5:58 PM
alangenfeld requested review of D7342: [mypy] environment_configs.py.
Wed, Apr 7, 5:29 PM
alangenfeld added a comment to D7261: [RFC] change DagsterInstance.get behavior.

requesting review to get more takes on release sooner vs later as well as on the user facing messages

Wed, Apr 7, 4:28 PM
alangenfeld requested review of D7261: [RFC] change DagsterInstance.get behavior.

I wasn't sure if this would be kosher for a minor release - I expect things shouldn't break but theres a small chance someone is depending on it. I could mention DagsterInstance.ephemeral() in the message.

Wed, Apr 7, 4:25 PM
alangenfeld closed D7340: pin against bad jupyter-client.
Wed, Apr 7, 4:01 PM
alangenfeld committed R1:ad6070c1f5ff: pin against bad jupyter-client (authored by alangenfeld).
pin against bad jupyter-client
Wed, Apr 7, 4:01 PM
alangenfeld published D7340: pin against bad jupyter-client for review.
Wed, Apr 7, 3:59 PM
alangenfeld added a reverting change for D7264: update scrapbook dep: D7339: Revert "update scrapbook dep".
Wed, Apr 7, 3:30 PM
alangenfeld added a reverting change for R1:55076f943f2e: update scrapbook dep: D7339: Revert "update scrapbook dep".
Wed, Apr 7, 3:30 PM
alangenfeld accepted D7327: Move run worker host mode code to its own module.

thanks

Wed, Apr 7, 3:26 PM
alangenfeld closed D7264: update scrapbook dep.
Wed, Apr 7, 2:44 PM
alangenfeld committed R1:55076f943f2e: update scrapbook dep (authored by alangenfeld).
update scrapbook dep
Wed, Apr 7, 2:44 PM
alangenfeld added inline comments to D7261: [RFC] change DagsterInstance.get behavior.
Wed, Apr 7, 2:22 PM

Tue, Apr 6

alangenfeld requested review of D7330: better error message for @resource param issues.
Tue, Apr 6, 10:23 PM
alangenfeld closed D6017: initial async solids support.
Tue, Apr 6, 9:17 PM
alangenfeld committed R1:56c206ab452d: initial async solids support (authored by alangenfeld).
initial async solids support
Tue, Apr 6, 9:17 PM
alangenfeld closed D7280: improve dynamic map & collect api docs.
Tue, Apr 6, 8:59 PM
alangenfeld committed R1:059255fe3f4a: improve dynamic map & collect api docs (authored by alangenfeld).
improve dynamic map & collect api docs
Tue, Apr 6, 8:58 PM
alangenfeld updated the diff for D6017: initial async solids support.

rebase, feedback

Tue, Apr 6, 8:35 PM
alangenfeld removed a reviewer for D6017: initial async solids support: schrockn.
Tue, Apr 6, 8:35 PM