Page MenuHomePhabricator

Change asset key storage structure, make asset key strings more permissive
ClosedPublic

Authored by prha on Oct 28 2020, 10:38 PM.

Details

Summary

Changes the asset key storage to be an JSON array instead of a custom
delimited string representation. Relaxes the AssetKey parameters to be
permissive instead of raising an Exception.

Could also consider a schema migration and supporting both a structured json
field as well as the legacy delimited field, so we don't have to do these OR sql
clauses (and deduping). That has a downside of forcing a migration, and so would
need to delay this until the 0.10.0 launch.

Test Plan

bk

Diff Detail

Repository
R1 dagster
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

prha requested review of this revision.Oct 28 2020, 10:57 PM
prha retitled this revision from RFC: Change asset key storage structure, make asset key strings more permissive to Change asset key storage structure, make asset key strings more permissive.Oct 28 2020, 10:58 PM
alangenfeld edited reviewers, added: sandyryza; removed: alangenfeld.
alangenfeld added a subscriber: sandyryza.

swapping out for @sandyryza to balance my review load, good luck!

I do think this requires a broader conversation about the future of the asset catalog. I'm having trouble evaluating this without further context

This seems like the most sensible thing we can do without introducing breaking changes.

This revision is now accepted and ready to land.Wed, Nov 4, 6:18 PM