Page MenuHomePhabricator

Allow solid subset selection using handles for nested solids
AbandonedPublic

Authored by max on Apr 25 2020, 7:02 PM.

Event Timeline

max created this revision.Apr 25 2020, 7:02 PM

Allow solid subset selection using handles for nested solids

huh how does this work if you select a single solid from within a composite? Wouldn't you need to reimplement _build_sub_pipeline for composites? I think this really hits home how busted that current approach is.

python_modules/dagster/dagster/core/definitions/solid.py
376

from a conceptual level it feels like we should have:
dependency_structure(as exists)
composition_structure (the structure you are building but maybe just str -> str)
solid_handle_dict (handle str -> solid instance)

these seem more fundamental and less bespoke

391

this name makes me think its just a map from handle string to solid instance - maybe descendants_by_handle ?

python_modules/dagster/dagster/core/definitions/solid_container.py
246–266

does this structure even make sense still? does this dict not overwrite if there are redundant aliases in disjoint composites?

273

to_string() here i think

max planned changes to this revision.Mon, Apr 27, 11:26 PM
max updated this revision to Diff 12710.Mon, Apr 27, 11:38 PM

Rebase on 2704

max planned changes to this revision.Mon, Apr 27, 11:38 PM
max planned changes to this revision.Thu, Apr 30, 12:26 AM
schrockn resigned from this revision.Thu, Apr 30, 5:55 PM

can you readd me when ready for review? trying to reduce queue thrash

alangenfeld requested changes to this revision.Fri, May 1, 10:17 PM
This revision now requires changes to proceed.Fri, May 1, 10:17 PM
max planned changes to this revision.Mon, May 4, 7:21 PM
max planned changes to this revision.Mon, May 4, 9:19 PM
max updated this revision to Diff 13169.Tue, May 5, 2:34 AM

rebase

max planned changes to this revision.Tue, May 5, 2:34 AM
max updated this revision to Diff 13239.Wed, May 6, 12:08 AM

rebase

max planned changes to this revision.Wed, May 6, 12:09 AM
max updated this revision to Diff 13473.Thu, May 7, 10:49 PM

rebase

max planned changes to this revision.Thu, May 7, 10:49 PM
max planned changes to this revision.Mon, May 11, 9:47 PM
max abandoned this revision.Wed, May 13, 1:47 AM