Page MenuHomeElementl

versioned_filesystem_io_manager requires base_dir
ClosedPublic

Authored by yuhan on Feb 23 2021, 6:57 AM.

Details

Summary

versioned_filesystem_io_manager requires the base_dir arg in the resource config

the alternative is not to require it but default base_dir to use the instance.storage_dir. rfc here D6636
i'm leaning towards this diff, see reasoning on D6636.

Test Plan

unit

Diff Detail

Repository
R1 dagster
Branch
yuhan/versioned-io-manager-dir
Lint
Lint Passed
Unit
No Test Coverage

Event Timeline

yuhan requested review of this revision.Feb 23 2021, 7:14 AM

Regarding resource initialization outside of execution; I do have a diff up moving in that direction (https://dagster.phacility.com/D6513). I do agree that it's probably better to default to requiring for now and let the dust settle on that, but my initial impression is that adding the instance would be kinda limiting. The failure cases on out of execution resource initialization are already kinda esoteric (ie my initialization requires run-specific information), and I don't feel like we fully understand that space yet.

This revision is now accepted and ready to land.Feb 23 2021, 4:15 PM