Page MenuHomePhabricator

Add initial sqllite run storage
ClosedPublic

Authored by schrockn on Aug 15 2019, 10:39 PM.

Details

Summary

This just reimplements the exact same run storage interface but with a sqlite backend. Very barebones

Test Plan

Buildkite. Included unit tests

Diff Detail

Repository
R1 dagster
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

schrockn created this revision.Aug 15 2019, 10:39 PM

@alangenfeld curious about what you think should be our standard for passing around connection strings given the py2-only bs you ran into.

schrockn updated this revision to Diff 3751.Aug 15 2019, 10:45 PM

py2 floats

schrockn updated this revision to Diff 3752.Aug 15 2019, 10:48 PM

py2 floats

schrockn updated this revision to Diff 3753.Aug 15 2019, 10:51 PM

self review

alangenfeld added inline comments.Aug 16 2019, 9:39 PM
python_modules/dagster/dagster/core/storage/runs.py
160–169

what do you think is good about the provider setup as opposed to just passing the file or ':mem:' to the constructor?

I suppose with this setup its actually SqlRunStorage since the only thing that knows its sqlite is the connection provider

schrockn updated this revision to Diff 3798.Aug 18 2019, 6:46 PM

conn instead of provider

schrockn added inline comments.Aug 18 2019, 7:24 PM
python_modules/dagster/dagster/core/storage/runs.py
160–169

So it doesn't work for :mem: because everytime you new up a new connection with ":mem:" it is totally new. So you need to hold onto the connection object.

alangenfeld accepted this revision.Aug 19 2019, 3:30 PM
This revision is now accepted and ready to land.Aug 19 2019, 3:30 PM
This revision was automatically updated to reflect the committed changes.