Page MenuHomePhabricator

Port build and push scripts to python
ClosedPublic

Authored by schrockn on Jul 2 2020, 10:01 PM.

Details

Summary

This is a port of the exiting build and push shell scripts to
python. I am working on adding another instance of an image and wanted
to reuse code and do more complicated things than shell scripts are good
for.

Test Plan

Run locally and manually

Diff Detail

Repository
R1 dagster
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

schrockn created this revision.Jul 2 2020, 10:01 PM
Harbormaster returned this revision to the author for changes because remote builds failed.Jul 2 2020, 10:14 PM
Harbormaster failed remote builds in B14637: Diff 17964!
schrockn requested review of this revision.Jul 2 2020, 10:52 PM
schrockn updated this revision to Diff 17969.Jul 2 2020, 11:03 PM
schrockn edited the summary of this revision. (Show Details)
schrockn added a reviewer: nate.

up

nate accepted this revision.Jul 5 2020, 5:36 PM

LGTM, although worth considering the Docker API since it should be a very easy change

.buildkite/images/integration/image_build.py
14

might be a good idea to use the Python Docker APIs for this instead of shelling out to docker? see https://docker-py.readthedocs.io/en/stable/images.html#docker.models.images.ImageCollection.build

python_modules/automation/automation/git.py
5

not for this diff, but we should move bin/git_utils.py to automation/ and use it here. maybe even move all of bin/ over since it's also automation.

This revision is now accepted and ready to land.Jul 5 2020, 5:36 PM
schrockn added inline comments.Mon, Jul 6, 7:51 PM
.buildkite/images/integration/image_build.py
14

this is surprisingly annoying to do since documentation is fairly poor and the rely on image/tag being separate which requires some non-trivial restructuing. going to call this a refactor we can do if there is well-defined value

This revision was automatically updated to reflect the committed changes.