i think this is a clear step forward even if its json sample in yaml land dagit

should any docs or tutorials get updated in this diff as well?

sweet thanks much

copy refresh - support dagster pipeline execute

oh wait - whats the testing story for IO managers? Do we expect users to create these by hand in test scenarios ? We might need to the typing.NamedTuple with __new__ version

switch the NamedTuples to still have __new__ but be typed

want @yuhan to sign off on this one

being annoying but i think its worth

land late tonight / early tomorrow for max soak between now and next release

😗 👌

typo

rebase, do some refactoring and leave useful comments explaning current setup

lint

just lint issues which i will fix

To be clear in vanilla solids you still have to raise these right?

• add comments / docblocks to the context classes explaining the thought in the current structure
• make sure to turn on integration tests too

definitely loving this direction - I think we can trim even further as highlighted with some inline comments

definitely worth adding comments to all the classes

seems reasonable to me

to your queue

I think it should be possible to update the test callsites to make sure a snapshot is recorded - likely to be a net improvement to the system depending on where you can enable it.

more tests and comments

ya let me take another pass at this

+ the system.py changes

quick drive by thoughts

• since hooks happen in process I think we should have the actual exception object available on failure
• I think we should probably have all the events from the step available, then maybe utilities for finding the success/failure hook from the full list

ExecutorProcessSetting giving you pause?

mypy only does static type checking, so we still need runtime checking. We still want to use check for validating values as they enter the program from external sources, like what users pass in or what we deserialize out of storage or from cross process communication.

requesting review to get more takes on release sooner vs later as well as on the user facing messages

I wasn't sure if this would be kosher for a minor release - I expect things shouldn't break but theres a small chance someone is depending on it. I could mention DagsterInstance.ephemeral() in the message.

thanks

rebase, feedback