Page MenuHomeElementl

Refactor public images to be substantially smaller

Authored by nate on Dec 19 2019, 12:20 AM.



Now built on python-alpine.

Pre: ~2GB. Post: 163MB.

Note that I do almost everything in a single RUN command to ensure that Docker layer stays small; splitting this up makes it easy to end up > 1GB

Test Plan

manually built & tested containers

Diff Detail

R1 dagster
No Lint Coverage
No Test Coverage

Event Timeline

What is the goal of having these image be public? Do we expect people who are not us to use them?

yeah, I'd anticipate we'll have people doing: FROM dagster/dagster-py37:latest as a base image, to layer their code on top of the latest Dagster.

This Dockerfile also serves as a good reference artifact to show users how to build their own, so they can start from this and write their own if for some reason they need to further customize how they install Dagster.

are you expecting users to host their own code and dagit from this?


I generally shy at this kind of surgery inside a Dockerfile, preferring to do it in the build context and using ADD to make it explicit what's in and what's out

Per in person discussion, we're going to do a couple things here:

(1) Remove publishing nightly Docker images; if we do publish public images, we should do so only for releases vs. the nightly build.
(2) Update all builds to use Python base images instead of the dagster public images

Abandoning for now in favor of D1749, but wanted to keep this diff preserved as a reference for the future if/when we decide to publish docker images in lockstep with our public releases.