Everything thats here seems reasonable from my perspective

One interesting problem that we've discussed in past iterations of this direction is how different @assets may materialize their input assets in to different in-memory types, controlled separately from how they are persisted. Thoughts on that? Something for a follow up?

@alangenfeld

One interesting problem that we've discussed in past iterations of this direction is how different @assets may materialize their input assets in to different in-memory types, controlled separately from how they are persisted. Thoughts on that? Something for a follow up?

I think this is a question even in the non-@asset world of solids and IOManagers. Our current recommendation is that, in their IOManager, users switch on the Python type attached to the InputDefinition. Here's an example in the hacker news pipeline: https://github.com/dagster-io/dagster/blob/master/examples/hacker_news/hacker_news/resources/parquet_io_manager.py#L46.

In D8969#234895, @sandyryza wrote:

@alangenfeld

One interesting problem that we've discussed in past iterations of this direction is how different @assets may materialize their input assets in to different in-memory types, controlled separately from how they are persisted. Thoughts on that? Something for a follow up?

I think this is a question even in the non-@asset world of solids and IOManagers. Our current recommendation is that, in their IOManager, users switch on the Python type attached to the InputDefinition. Here's an example in the hacker news pipeline: https://github.com/dagster-io/dagster/blob/master/examples/hacker_news/hacker_news/resources/parquet_io_manager.py#L46.

I'm still pretty pro-io_manager_key in the @asset definition, mostly because of these analogs between the non-asset world and the asset world. Theoretically, this same exact scheme of having a single mono-IOManager fo all inputs/outputs would work essentially the same way in the non-asset world, so it seems a bit weird to force this pattern for the asset use cases and not even encourage it for the others. When I was refactoring the hacker_news IOManagers, the pattern of one IOManager per storage medium seemed to work quite well in terms of keeping things organized and understandable (and made it easy to swap out specific parts of the logic between different modes).

There's definitely some elegance to a pattern that allows writers of @assets to forget that io_managers exist at all, and having the mono-IOManager figure out what to do purely based off of types / metadata (especially for less technical users), but having an io_manager_key parameter seems purely additive (after all, the default io_manager_key is 'io_manager' -- if users want to go down that route, there's nothing stopping them).

TLDR; users familiar w/ io_manager_key patterns for their solids will likely miss them if they try out @assets, and people who want to have a single mega IOManager should be unaffected by the existence of an io_manager_key parameter.

@owen - I updated the diff to include io_manager_keys

I added docstrings and beefed up the tests - I think this is ready to go in for others to build on top of. It'll be hidden behind dagster.assets, so still not part of the public API.