i think this is a good place to start
Thu, May 13
maybe add a DagsterInvalidInvocationError for when the wrong info is passed?
Wed, May 12
Tue, May 11
these pretty safe to me - hard to imagine how theyll be a compat / maintenance issue
would be good to have the multi output test have direct invoke right next to execute_pipeline / execute_solid to demonstrate their alignment (or lack thereof)
Mon, May 10
++ sandy's feedback
- more tests
Fri, May 7
il defer to @sandyryza here on how exactly to frame whether this is deprecated, legacy, or whatever
I support landing this to maximize soak time til next release
Thu, May 6
How would this work in a world where we encourage people to print (or whatever) instead of using context.log?
can resolve my error concerns in a follow up - your call
would like @prha to sign off on this too
That seemed much worse than what I think you're proposing, which is broadening what the outputs of sensors are and relaxing the constraints of what they can kick off (which would strictly be a pipeline run)
would love to get this in the release today
This is behavior that is due to the difference between websockets & http - the websocket response does not have an 'error' field on the JS side so the components render differently. Not sure if throwing away the error in this case (to conform to the old behavior) makes sense though -- feels like errors are important to pay attention to!
Wed, May 5
I think we should strongly consider making hooks standalone objects on the repository, like sensor defs, rather than live on pipeline definitions.
Tue, May 4
ill put this in my queue for now accordingly
to your queue for docs/error message pass
_context and context_ are such a minor corner case that im not worried - I think this is basically good to go
I think im cool with this once you can get the tests passing
would be good to include mypy return types as well
Mon, May 3
the big thing you lose there is config playground typeahead support in dagit right?
heres a quick prototype to illustrate the idea from previous comment https://dagster.phacility.com/differential/diff/36671/
@sandyryza brought up an interesting kernel of an idea about just making the execution block of config Permissive - at least from the pipeline perspective.
So you could drop threading the default executors to the snapshot and env configs and let those just have empty executors and default to a Permissive section. Then only at the "real" execution call sites resolve the instance executors and maybe do a separate config validation/processing pass for the execution subsection
One case i thought of that I think communicates the cost of this change well is if we had a function for just doing run config validation against a target - it would now need to take the instance as an argument
check_run_config(my_pipeline, run_config, instance=DagsterInstance.get())
which gets a little hairy if one wanted to put said function in their test suite to ensure their production runs config was expected to be valid. My assumption here is that generally the test environment will not be setup with the same instance. There are a few different ways to deal with this so it's not a hard blocker - just an awkward case that makes me feel like this isn't the cleanest modeling of this complexity.
Fri, Apr 30
to your queue for test suite
I think it would be useful to add a block of tests that clearly communicate the new setup by having the different permutations all next to each other
some ideas inline - I think this can use a second pass. Its a little better but not a clear step up imo
Thu, Apr 29
open questions / thinking out loud: