HomeElementl

RFC: fix result.result_for_solid when using mem_io_manager + default to…

Description

RFC: fix result.result_for_solid when using mem_io_manager + default to mem_io_manager

Summary:
this diff does two things

  1. https://github.com/dagster-io/dagster/issues/3508

    RFC: because io managers are resources, the default mem_io_manager won't work for things like result.result_for_solid. it's because [2] output_value will reconstruct the pipeline context, where resources get re-initiated, which means we would lose the in-memory values in the mem_io_manager as the io_manager in output_value is going to be a brand new instance of the InMemoryioManager.
  1. default to mem_io_manager

    switches the logic to check "is default intermediate storage" instead of "is default io manager" in the adapting logic - so we would be using mem_io_manager as the default rather than the adapted mem intermediate storage.

couldn't separate these two because this check will always switch to use intermediate storage if mem_io_manager is used

Test Plan: bk

Reviewers: sandyryza, cdecarolis, schrockn

Reviewed By: schrockn

Subscribers: schrockn

Differential Revision: https://dagster.phacility.com/D5834

Details

Provenance
yuhanAuthored on Jan 6 2021, 8:25 AM
Reviewer
schrockn
Differential Revision
D5834: RFC: fix result.result_for_solid when using mem_io_manager + default to mem_io_manager
Parents
R1:e30868ea2503: [dagit] Link to run config from Queued tab
Branches
Unknown
Tags
Unknown