Page MenuHomePhabricator

Refactor public images to be substantially smaller

Authored by nate on Dec 19 2019, 12:20 AM.



Now built on python-alpine.

Pre: ~2GB. Post: 163MB.

Note that I do almost everything in a single RUN command to ensure that Docker layer stays small; splitting this up makes it easy to end up > 1GB

Test Plan

manually built & tested containers

Diff Detail

R1 dagster
No Linters Available
No Unit Test Coverage

Event Timeline

nate updated this revision to Diff 7952.Dec 19 2019, 12:20 AM
nate created this revision.


nate edited the summary of this revision. (Show Details)Dec 19 2019, 12:24 AM
nate added reviewers: schrockn, max.

What is the goal of having these image be public? Do we expect people who are not us to use them?

Harbormaster completed remote builds in B6436: Diff 7952.
nate added a comment.Dec 19 2019, 12:32 AM

yeah, I'd anticipate we'll have people doing: FROM dagster/dagster-py37:latest as a base image, to layer their code on top of the latest Dagster.

This Dockerfile also serves as a good reference artifact to show users how to build their own, so they can start from this and write their own if for some reason they need to further customize how they install Dagster.

nate updated this revision to Diff 7954.Dec 19 2019, 12:39 AM

remove extraneous command

are you expecting users to host their own code and dagit from this?

max added inline comments.Dec 19 2019, 3:00 AM

I generally shy at this kind of surgery inside a Dockerfile, preferring to do it in the build context and using ADD to make it explicit what's in and what's out

nate planned changes to this revision.Dec 19 2019, 5:54 PM

Per in person discussion, we're going to do a couple things here:

(1) Remove publishing nightly Docker images; if we do publish public images, we should do so only for releases vs. the nightly build.
(2) Update all builds to use Python base images instead of the dagster public images

nate abandoned this revision.Dec 28 2019, 9:54 PM

Abandoning for now in favor of D1749, but wanted to keep this diff preserved as a reference for the future if/when we decide to publish docker images in lockstep with our public releases.