HomePhabricator

ObjectStoreOperation/AssetStoreOperation -> HandledOutput/LoadedInput

Description

ObjectStoreOperation/AssetStoreOperation -> HandledOutput/LoadedInput

Summary:
RIP AssetStoreOperation and ObjectStoreOperation in event logs.

This diff does several things

  • introduces HandledOutput/LoadedInput to log the completion of IO manager's handle_output/load_input operations (previously in D5519 which is a bit hard to test separately without the core changes, so i moved it back to this diff)
  • completely gets rid of AssetStoreOperation which is replaced by HandledOutput/LoadedInput. https://github.com/dagster-io/dagster/issues/3509
  • no long emits ObjectStoreOperation for GET/SET objects in the event logs, notes:
    • ObjectStoreOperation as an event definition still exists because it would be yielded in the user code when a user write their own Intermediate Storage
    • in order to get rid of ObjectStoreOperation in the event logs and (kinda) avoid regression like https://github.com/dagster-io/dagster/issues/3368, this diff implements a stopgap solution to context.log.info the address of an intermediate object see [1]

Test Plan:
bk

~/dev/dagster/examples/docs_snippets/docs_snippets/overview/io_managers
dagster-3.7.4 $ dagit -p 3333 -f default_io_manager.py

io manager:

intermediate storage:

Reviewers: sandyryza, schrockn, alangenfeld

Reviewed By: sandyryza

Differential Revision: https://dagster.phacility.com/D5474

Details

Provenance
sandyryzaAuthored on Tue, Jan 12, 10:54 PM
yuhanCommitted on Tue, Jan 12, 10:56 PM
Reviewer
sandyryza
Differential Revision
D5474: ObjectStoreOperation/AssetStoreOperation -> HandledOutput/LoadedInput
Parents
R1:a2c3c566cde4: [docs] Eliminate 'intermediates' in docs #3516
Branches
Unknown
Tags
Unknown