The description for the resources_strict_mode boolean flag
on PipelineDefinition describes this behavior.
resources_strict_mode (Optional[Bool]): Defaults to false. If true this uses to
required_resource_keys to provide a Resources objects in the context with *only*
those keys. If the user does not specify required_resource_keys in a solid in a
pipeline in strict mode, no resources are provided to the context. This mode
allows us to be more selective about what resources are spun up for different
pipeline subset executions. For example, without this mode a pipeline with
an expensive resource (such as a Spark context) will spin up the context
even if the current pipeline or execution plan subset does not require it. In
strict mode, all solids are forced to declare their resource dependencies
and therefore only those resources will be booted up.
This boolean is meant to be temporary until all tests are migrated. We
want all solids to have this strict behavior.I would propose that the goal would be to have this strict mode actually
be the only behavior. This would add friction to the first usage of
resources. However it is really intuitive the moment you start doing
any out-of-process execution or pipeline subselection.
One thing this does not currently cover are types and plugins within
types that require resources. Once overall approach here is vetted
can add that.
Depends on D1501