Page MenuHomePhabricator

[docker] Improve caching in editable images
AbandonedPublic

Authored by sashank on Oct 21 2020, 2:56 PM.

Details

Summary

This diff separates out an editable dagster install into its own last layer. This significantly speeds up Docker builds when only editing dagter core.

Test Plan

bk

Diff Detail

Repository
R1 dagster
Branch
more-image-optimizations
Lint
Lint OK
Unit
No Unit Test Coverage

Event Timeline

Does it not matter that the other libraries depend on dagster core?

alangenfeld added a subscriber: catherinewu.

when only editing dagter core

is this the case we want to optimize for?

defer to other folks - maybe @catherinewu has thoughts

i think building images to make dagster core changes is not the most optimal dev loop?

i think building images to make dagster core changes is not the most optimal dev loop?

Yeah you're right, but I found myself needing to do this so I thought I'd push it up. Happy to abandon.

Does it not matter that the other libraries depend on dagster core?

Hm I think the base image will have the latest dagster version pulled in from pypi, but the editable install is layered on top in the last step

Hm I think the base image will have the latest dagster version pulled in from pypi, but the editable install is layered on top in the last step

Still kinda confused here, the base image is just python-slim right? Don't have a strong opinion either way

johann added a subscriber: johann.