A set-up change for direct targeting, orients the current set-up around "repo relative" targets of string pointers
Details
Diff Detail
- Repository
- R1 dagster
- Lint
Lint Not Applicable - Unit
Tests Not Applicable
Event Timeline
Interesting. Are there different combinations of things that are potentially supported here / could you partially specify? Or is the idea that in a post-crag world, all of these things would be coupled into a singular entity?
Are there different combinations of things that are potentially supported here / could you partially specify? Or is the idea that in a post-crag world, all of these things would be coupled into a singular entity?
I think most likely the singular entity - the diffs above will allow
@schedule( target=my_graph.with_resources(...), ...
Yup, I like using target as a singular noun here. Way easier to think about mode/solid selection.
In the case we end up supporting one-to-many sensors to targets, what would the run request look like? Would we consider exposing RepoRelativeTarget and force users to construct them? Or should the targets themselves have names?
In the case we end up supporting one-to-many sensors to targets, what would the run request look like?
my current idea
@sensor( targets={'foo': my_graph.with_resources(...), 'bar': other_graph.with_resources(...)} ... ) def ok(): yield RunRequest( target_key='foo', ... )
going with dict/key approach since you may have the same graph with different resources which makes targeting by name likely awkward / tricky
Would we consider exposing RepoRelativeTarget and force users to construct them?
maybe - you would likely only use it on conjunction with lazy-loaded-pipelines (targets) in your repo definition which is an advanced perf optimization kinda thing
Or should the targets themselves have names?
Im open to the idea of accepting target="graph_name", target="graph_name[qualifier_name]", or target=("graph_name", "qualifier_name") instead of having to construct a dingus but also respect the arguments against super flexible input typing