HomeElementl

Allow classes to be decorated with @experimental

Description

Allow classes to be decorated with @experimental

Summary:
The existing @experimental decorator doesn't work particularly well when
trying to decorate a class. Previously, for vanilla class decorations,
things would mostly work:

@experimental
class Foo:
  pass

foo = Foo()

would emit:

ExperimentalWarning: "Foo" is an experimental function.

That error is mostly correct except for the fact that Foo is supposed
to be an experimental class and not an experimental function. But
it's more than just a copy error - because the @experimental decorator
is returning an _inner function, Foo truly is of type function.

The most immediate consequence is that nothing can properly subclass
Foo:

@experimental
class Bar(Foo):
  pass

fails with:

E   TypeError: function() argument 'code' must be code, not str

It's a bit of a cryptic error but it's throwing because you can't
subclass a function. https://bugs.python.org/issue6829

An interesting Dagster-specific consequence is that you can't mark
anything that subclasses ConfigurableClass as @experimental:

@experimental
class Configurable(ConfigurableClass):
  ...

fails with:

E       TypeError: issubclass() arg 1 must be a class

It's throwing because issubclass expects its first argument to
be a class and we're giving it a function:

https://github.com/dagster-io/dagster/blob/34bfa587ffb19821a272f76c52dfb750e0ca7de8/python_modules/dagster/dagster/serdes/config_class.py#L70

One option is to take the approach that we took when introducing
@experimental_decorator - we could add a new @experimental_class and
leave it up to the author to correctly decide when to decorate with
@experimental and when to decorate with @experimental_class.

Instead, I've decided to conditionally switch on whether the callable
being decorated is a function or a class. If it's a function, we
do the same wrapping behavior we've always done. If it's a class, we
augment __init__ to also include an experimental_class_warning but
we still return a class.

Test Plan: unit

Reviewers: sandyryza

Reviewed By: sandyryza

Differential Revision: https://dagster.phacility.com/D8484

Details

Provenance
jordansandersAuthored on Jun 21 2021, 4:15 PM
Reviewer
sandyryza
Differential Revision
D8484: Allow classes to be decorated with @experimental
Parents
R1:34bfa587ffb1: correctly process configs of io managers on solids inside composite
Branches
Unknown
Tags
Unknown