Page MenuHomePhabricator

Use hashed names for all system dictionaries and selectors

Authored by schrockn on Dec 6 2019, 5:19 PM.



This diff eliminates names for the our system dictionary
configs. It turns out we can reuse dictionary structures and, in large
dags, this can be substantial. We are seeing 4x reducing in type
generation a commensurate improvement in config editor perf.

Test Plan

BK. Load large schema. See 4x reduction in instatiated types.

Diff Detail

R1 dagster
Automatic diff as part of commit; lint not applicable.
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

schrockn created this revision.Dec 6 2019, 5:19 PM
schrockn updated this revision to Diff 7167.Dec 6 2019, 6:05 PM
schrockn retitled this revision from Use hashed names for all system dictionaries to Use hashed names for all system dictionaries and selectors.
schrockn edited the summary of this revision. (Show Details)
schrockn edited the test plan for this revision. (Show Details)
schrockn added reviewers: max, alangenfeld.


max accepted this revision.Dec 6 2019, 6:16 PM
max added inline comments.

not immediately obvious to me that this issue is related

This revision is now accepted and ready to land.Dec 6 2019, 6:16 PM
schrockn added inline comments.Dec 6 2019, 6:37 PM

nor is it to me, however it was exposed here for some reason