Change solids.py::define_dagstermill_solids to allow for custom descriptions & tags. Default tags & descriptions are the same as before, unless explicitly written over by user input. See https://github.com/dagster-io/dagster/issues/3290 for more context.
Added tests for custom/default tags & descriptions, respectively. For tags, checked that 1) default tags were present (didn't test overwriting them bc. I can't imagine a case where the user would want to change the "kind" : "ipynb" mapping for a notebook, for example). I also checked 2) that custom tags were present when added. For descriptions, I checked that the default description was present unless a custom description was passed in.
We should document this decision in the doc string. We can also check the tags dict w/ something like the following:
check.invariant(notebook_path not in tags, 'user-defined solid tags contains notebook_path key, but notebook_path key is reserved for use by dagster')
Interesting. This works, and I think we should test the solid tags directly as well (easier to read + maintain)
Fix test structure & check that default tags are not overridden
Tests: Use define_dagstermill_solid function directly to simplify test structure; get rid of old pipeline tests (except for one to make sure that pipelines still run with custom tags & descriptions).
Tags: use check library to make sure that default tags ('kind' and 'notebook_path' are not overridden by the user)
Awesome, looks great. one last change (and a missing .) and then it's good to ship
we should use validate_tags which both checks the opt_dict_param part but also verifies that the json can be stringified.
from dagster.core.definitions.utils import validate_tags