Page MenuHomeElementl

[Asset Store Versioning 2 of 3] Added VersionedAssetStore abstract class and filesystem versioned asset store.
ClosedPublic

Authored by cdecarolis on Nov 10 2020, 5:21 PM.

Details

Summary

Added an abstract class to handle storing versioned assets. This way the core API doesn't change, and doesn't require much plumbing on behalf of the user since the API will find the version itself.

Test Plan

Added new unit tests to AssetStore

Event Timeline

sandyryza added inline comments.
python_modules/dagster/dagster/core/storage/asset_store.py
274

Maybe remove this line?

It would be helpful to include a line explaining that the version will be part of the addressing scheme.

This revision is now accepted and ready to land.Nov 16 2020, 7:23 PM

fixed messaging on filesystem_versioned_asset_store

Migrated to using asset store context

python_modules/dagster/dagster/core/storage/asset_store.py
264

We should call this versioned_fs_asset_store for consistency

Removed VersionedAssetStore abstract class, and instead added a has_asset method to the base assetstore class.

python_modules/dagster/dagster/core/storage/asset_store.py
65

nit?

182

maybe this can be moved to _get_path? your call.

196

should we pass context instead?

Fixed nits. Re-added VersionedAssetStore abstract class so that users are not forced to implement has_asset method