This enables asset stores to define a config schema that applies to each output. This enables us to support the output-related user stories described in https://elementl.quip.com/scopA6J0p5R3/InputOutput-Parameterization.
An assumption backing this direction is that there's a spectrum between "pure config-driven output storing" and "pure fixed location-driven output storing", and that users are likely to want to do things in the middle.
- Pure config-driven output storing would be something like "store the output of solid X using the format and path specified at runtime by the analyst".
- Pure fixed location-driven output storing would be something like "always store the output of solid X in table Y".
- "Something in the middle" would be "store the output of solid X in a file with a name given at runtime by the analyst, using a directory and file format determined by the pipeline author".
This change aims at:
- Making it more reasonable to support the third bullet.
- Moving towards a world in which there's a single, user-definable code path for storing outputs, instead of type materializers vs. asset stores.
- Moving towards the same for inputs.
Particularly wanted eyes on some of the controversial bits:
- AssetStoreDefinitions that extend ResourceDefinitions.
- Solid-level config that depends on what resource definitions are selected in the mode.
I don't like the name "store_outputs". Just a placeholder until we think of something better or consolidate with "outputs".
Take a look at test_asset_store_with_config for an example.