Page MenuHomeElementl

Do Postgres/MySQL table creation in a transaction
ClosedPublic

Authored by dgibson on Jun 7 2021, 2:55 PM.
Tags
None
Referenced Files
Unknown Object (File)
Sun, Jun 26, 3:27 AM
Unknown Object (File)
Sun, Jun 19, 2:14 PM
Unknown Object (File)
Sun, Jun 19, 9:25 AM
Unknown Object (File)
Sat, Jun 18, 5:54 AM
Unknown Object (File)
Sat, Jun 18, 3:55 AM
Unknown Object (File)
Fri, Jun 17, 10:40 PM
Unknown Object (File)
Fri, Jun 10, 10:47 AM
Unknown Object (File)
Fri, Jun 10, 5:52 AM
Subscribers
None

Details

Summary

User asked why we don't do this and I didn't have a great answer. Let's do the table creation and alembic writes all in one big transaction.

(There is still some jankiness from the fact that we share alembic configs between the three different storage classes - ideally there would be some way to do the full table creation all in one transaction, but this is still an improvement I think)

Test Plan

BK

Diff Detail

Repository
R1 dagster
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

Harbormaster returned this revision to the author for changes because remote builds failed.Jun 7 2021, 3:28 PM
Harbormaster failed remote builds in B31751: Diff 39100!

something about doing this in a transaction now made sqlalchemy complain about not setting an index length on some indices in mysql, picked a reasonable-looking default. Do I need to add these as a migration? (The index isn't new, just hte length restriction)

This revision is now accepted and ready to land.Jun 8 2021, 8:34 PM