- User Since
- Jul 20 2020, 3:42 PM (32 w, 3 d)
One thing that I'm concerned about is communicating the failure modes of the resolution mechanism that we're doing here. It's a bit of magic, and there's a lot of ways each can fail.
Removed comment, removed recorder
Added docstring, init_resources -> build_resources
Wed, Mar 3
Addressing comments + propogating changes
Don't provide default values for resource initialization
Get rid of refactor artifact
Tue, Mar 2
Pull change into dagstermill
Fixed arg on InitResourceContext
Split up diff
Mon, Mar 1
Fixed tests to account for new resource mechanism
Fri, Feb 26
Added ResourceConfig wrapper
Thu, Feb 25
Major refactor, reuse most of core resource init path
Tue, Feb 23
Regarding resource initialization outside of execution; I do have a diff up moving in that direction (https://dagster.phacility.com/D6513). I do agree that it's probably better to default to requiring for now and let the dust settle on that, but my initial impression is that adding the instance would be kinda limiting. The failure cases on out of execution resource initialization are already kinda esoteric (ie my initialization requires run-specific information), and I don't feel like we fully understand that space yet.
Fix run config passed in to init_resources from version resolution pathway
Mon, Feb 22
Got rid of InitResourceContext artifacts, accept run config instead of env config
This is no longer needed.
Fri, Feb 19
Added recursive output structure.
Thu, Feb 18
if we communicate it well in a release do you think that's enough to warrant breaking?
Maybe reconsider that one error message, but lgtm
Wed, Feb 17
Tue, Feb 16
gonna iterate on this - not requiring an execution plan or run id.
Mon, Feb 15
Fri, Feb 12
Thu, Feb 11
Passed through callback fxn at top level instead of pure dict.
Will take a closer look tomorrow, but the idea feels good to me. A good before-after would of course be the snowflake IO manager on internal.
Wed, Feb 10
Needs a little more juice to work for graphs, but the main idea is working for solids.
Tue, Feb 9
Looped in adding a property for better error state when solid/graph does not have output value.