Page MenuHomePhabricator

Don't always run create_all during storage creation, add stamps to postgres storage
ClosedPublic

Authored by dgibson on Jan 13 2021, 10:49 PM.

Details

Summary

Still testing this, but this is a more invasive migration solution that should leave our alembic stamps consistent .

Instead of relying on create_all to create the new tables we added in 0.10.0, we only create new tables during the initial load, and during migration.

Test Plan

BK

Diff Detail

Repository
R1 dagster
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

remove a test involving migrating from a time when there was no ennet log, since that was incorrectly stamping (is it safe to assume that 0.7.6 => 0.10.0 migrations are not a thing?)

Harbormaster returned this revision to the author for changes because remote builds failed.Jan 13 2021, 11:16 PM
Harbormaster failed remote builds in B24332: Diff 29612!

keep the skip_migration_checks stuff out of DagsterInstance.get()

result of chat with phil - take out places where we were dropping the "schedules" table in past migrations, so that we don't lose our clue in the 0.10.0 migration that we're in the schedule DB. Instead, drop the schedules table at the end of hte 0.10.0 migration.

python_modules/dagster/dagster/core/storage/schedules/sqlite/alembic/versions/140198fdfe65_wipe_schedules_table_for_0_10_0.py
38–39

maybe we should remove these too.

python_modules/dagster/dagster/core/storage/schedules/sqlite/alembic/versions/b22f16781a7c_.py
27–28

maybe comment these out instead of removing? and add a comment about why?

python_modules/libraries/dagster-postgres/dagster_postgres/run_storage/alembic/versions/5c18fd3c2957_scheduler_update.py
27–28

same, maybe comment here instead?

This revision is now accepted and ready to land.Jan 14 2021, 4:53 AM
This revision was landed with ongoing or failed builds.Jan 14 2021, 3:22 PM
This revision was automatically updated to reflect the committed changes.