Page MenuHomeElementl

piped output metadata up to the logs
Changes PlannedPublicDraft

Authored by owen on Feb 8 2021, 11:52 PM.

Details

Reviewers
sandyryza
Summary

This is just a quick diff that shows us attaching metadata entries to an Output and representing them in Dagit.

The idea behind this is to make the ability to attach metadata entries to Outputs useful regardless of when

Test Plan

n/a

Diff Detail

Repository
R1 dagster
Branch
output_metdata_exploration
Lint
Lint Warnings
SeverityLocationCodeMessage
Warningpython_modules/dagster/dagster/core/execution/plan/inputs.py:122W0613Unused Argument
Unit
No Test Coverage

Event Timeline

Harbormaster returned this revision to the author for changes because remote builds failed.Feb 9 2021, 12:58 AM
Harbormaster failed remote builds in B25463: Diff 31064!
  • fixed linting issues, made example slightly more realistic

Screen Shot 2021-02-08 at 5.24.58 PM.png (518×1 px, 114 KB)
sample of what this looks like in dagit

Harbormaster returned this revision to the author for changes because remote builds failed.Feb 9 2021, 1:41 AM
Harbormaster failed remote builds in B25466: Diff 31067!
  • testable version of attaching metadata to AMs, still a bit grimy
Harbormaster returned this revision to the author for changes because remote builds failed.Feb 11 2021, 9:11 PM
Harbormaster failed remote builds in B25663: Diff 31321!
python_modules/dagster/dagster/core/execution/plan/execute_step.py
425

Nitpick: we can use f-strings now

445

If I understand correctly, the assumption here is that there will only be a single top-level asset key, with potentially multiple partitions? If that's the case, it likely makes sense to encode that into the object that we expect from the user. E.g. get_output_asset_keys (maybe rename?) could return an object called AssetPartitions or something with contains a single AssetKey and a list of partitions?

462

Nitpick, we generally try to avoid these kinds of abbreviations. Maybe result and metadatas_or_materializations?

465

Do we need to hang on to a list of these? Can we just yield them directly?

  • testable version of attaching metadata to AMs, still a bit grimy
Harbormaster returned this revision to the author for changes because remote builds failed.Feb 12 2021, 1:10 AM
Harbormaster failed remote builds in B25695: Diff 31360!
  • wip
  • surface parent asset keys in dagit
Harbormaster returned this revision to the author for changes because remote builds failed.Feb 16 2021, 1:08 AM
Harbormaster failed remote builds in B25855: Diff 31562!