Page MenuHomeElementl

Refactor public images to be substantially smaller
AbandonedPublic

Authored by nate on Dec 19 2019, 12:20 AM.

Details

Reviewers
schrockn
max
Summary

Now built on python-alpine.

Pre: ~2GB. Post: 163MB.

Note that I do almost everything in a single RUN command to ensure that Docker layer stays small; splitting this up makes it easy to end up > 1GB

Test Plan

manually built & tested containers

Diff Detail

Repository
R1 dagster
Branch
public_images
Lint
No Lint Coverage
Unit
No Test Coverage

Event Timeline

What is the goal of having these image be public? Do we expect people who are not us to use them?

yeah, I'd anticipate we'll have people doing: FROM dagster/dagster-py37:latest as a base image, to layer their code on top of the latest Dagster.

This Dockerfile also serves as a good reference artifact to show users how to build their own, so they can start from this and write their own if for some reason they need to further customize how they install Dagster.

are you expecting users to host their own code and dagit from this?

.buildkite/images/docker/Dockerfile.public
24

I generally shy at this kind of surgery inside a Dockerfile, preferring to do it in the build context and using ADD to make it explicit what's in and what's out

Per in person discussion, we're going to do a couple things here:

(1) Remove publishing nightly Docker images; if we do publish public images, we should do so only for releases vs. the nightly build.
(2) Update all builds to use Python base images instead of the dagster public images

Abandoning for now in favor of D1749, but wanted to keep this diff preserved as a reference for the future if/when we decide to publish docker images in lockstep with our public releases.