Just to give you guys a sense of what this will look like. It is truly unfortunate. However SolidInvocation is exported top-level so we will need a story.
The more obvious structure:
class NodeInvocation(namedtuple("Solid", "name alias tags hook_defs")): def __new__(cls, name, alias=None, tags=None, hook_defs=None): name = check.str_param(name, "name") alias = check.opt_str_param(alias, "alias") tags = frozentags(check.opt_dict_param(tags, "tags", value_type=str, key_type=str)) hook_defs = frozenset(check.opt_set_param(hook_defs, "hook_defs", of_type=HookDefinition)) return super(cls, NodeInvocation).__new__(cls, name, alias, tags, hook_defs) class SolidInvocation(NodeInvocation): """Identifies an instance of a solid in a pipeline dependency structure. Args: name (str): Name of the solid of which this is an instance. alias (Optional[str]): Name specific to this instance of the solid. Necessary when there are multiple instances of the same solid. tags (Optional[Dict[str, Any]]): Optional tags values to extend or override those set on the solid definition. hook_defs (Optional[Set[HookDefinition]]): A set of hook definitions applied to the solid instance. Examples: .. code-block:: python pipeline = PipelineDefinition( solid_defs=[solid_1, solid_2] dependencies={ SolidInvocation('solid_1', alias='other_name') : { 'input_name' : DependencyDefinition('solid_1'), }, 'solid_2' : { 'input_name': DependencyDefinition('other_name'), }, } ) In general, users should prefer not to construct this class directly or use the :py:class:`PipelineDefinition` API that requires instances of this class. Instead, use the :py:func:`@pipeline <pipeline>` API: .. code-block:: python @pipeline def pipeline(): other_name = solid_1.alias('other_name') solid_2(other_name(solid_1)) """ def __new__(cls, name, alias=None, tags=None, hook_defs=None): name = check.str_param(name, "name") alias = check.opt_str_param(alias, "alias") tags = frozentags(check.opt_dict_param(tags, "tags", value_type=str, key_type=str)) hook_defs = frozenset(check.opt_set_param(hook_defs, "hook_defs", of_type=HookDefinition)) return super(cls, SolidInvocation).__new__(cls, name, alias, tags, hook_defs
Results in the unfortunate error:
python_modules/dagster/dagster/core/definitions/dependency.py:153: in __new__ return super(cls, SolidInvocation).__new__(cls, name, alias, tags, hook_defs) python_modules/dagster/dagster/core/definitions/dependency.py:112: in __new__ return super(cls, NodeInvocation).__new__(cls, name, alias, tags, hook_defs) E TypeError: super(type, obj): obj must be an instance or subtype of type
So the solution I came up was to have an inheritance hierarchy and then have it
contain a namedtuple. I did containment to ensure maintainence precisely the same
hashing and equality semantics. (Side note: hooks seem problematic here)
I'd love to have a better solution here so I'm all ears
Depends on D4912