Page MenuHomePhabricator

Use WAL for sqlite-backed run storages as well
ClosedPublic

Authored by max on Dec 19 2019, 7:53 PM.

Diff Detail

Repository
R1 dagster
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

test plan Manual

should we use the backfill stuff from @prha to verify that this makes a difference?

given the reading I've done I believe this change is fine at a fundamental level - so I'm torn on whether to do the investigation on whether it address our issues here at diff time or not.

I will defer to you on sequencing

This revision is now accepted and ready to land.Dec 19 2019, 8:20 PM

can't repro @prha's backfill issue -- including before the WAL change on the event logs

but @prha can still repro

to be clear he can repro with this diff patched?

looking at https://github.com/dagster-io/dagster/issues/2021 too many open files is not an issue this would address - this would only fix lock contention