- User Since
- Jul 20 2020, 3:42 PM (9 w, 1 d)
Looks like this is passing in bk - not sure why it's saying it failed.
Used string method by convention
Added return to docstring
This looks largely good, although I think it would also be affected by https://dagster.phacility.com/D4480. Would you be able to refactor to take that into account?
I think a lot of the most recent changes to https://dagster.phacility.com/D4480 will break this, unfortunately. Most notably the renaming of the config_value parameter,
Renamed config_value to run_config, added an additional test for resourced solids.
Added tests to test_required_resources.py to ensure that default version was correct, and version was being passed all the way to resource definition.
Fri, Sep 18
@sandyryza I needed to make one change to create_run_for_pipeline.py - want to make sure this doesn't screw up what you were working on.
Made helper function to avoid directly accessing _composition_stack global.
Thu, Sep 17
I actually think we should do the opposite; there is no convenient way of obtaining mode or environment_config, both of which will be necessary in order to resolve config + resource versions.
This makes sense. It was kind of wonky having both options, and added a lot of unnecessary complication.
This approach makes sense to me. However, I do feel like some of these helper functions should probably live in some util file or something (like join_and_hash, etc)
Wed, Sep 16
Changed default value of external input version loader to None. Added experimental warning for _external_version_fn
Made warning message easier to read. Improved tests. Pending invocations tracked at the composition context level.
Fxn naming changes. Documentation.
Tue, Sep 15
Combined hash and join helper methods. Moved resolution of step versions to its own function on the dagster instance. Expanded testing to verify correct versions being produced.
Mon, Sep 14
Added public loader_version property.
Made loader version a public property
Changed parameter to solid_version. Refactored tags and hooks to use solid.
Good stuff. Makes sense to me.
Fri, Sep 11
@alangenfeld just to provide some context, I use this to pass version information from solid when computing the version of a step. I figured since solid was already supplied as an input to the __new__ method, this was the least intrusive way to do that. Additionally since ExecutionSteps aren't persisted, it wouldn't affect run storage, etc.
Made it explicit that "no version provided" sentinel is the None value
Added tests, and made default computed version None (we're using None as our sentinel value for no versions provided).
Checked that tags exist
Thu, Sep 10
Wed, Sep 9
Rebased onto master
Made StepInput version computed, and propagated up to ExecutionStep version as a result.
Updated to take config_value, and not context parameter
Updated to reflect changes to DagsterTypeLoader. Reformatted via make black.
Added documentation. Changed version parameter to loader_version to reflect its usage more effectively. Reflected that external_version_fn will take in extra parameters. Rebased onto master.
Tue, Sep 8
Implementation for hooks and tags, more comprehensive tests. Flushing of stack upon checking whether invoked.
Fri, Sep 4
More descriptive error messages, accounting for tags and hooks.
Thu, Sep 3
Wed, Sep 2
Flakey build failures.