Page MenuHomeElementl

Use the EcsRunLauncher in the reference deployment

Authored by jordansanders on Jun 23 2021, 2:56 PM.
Referenced Files
F2982867: D8492.diff
Sun, May 28, 9:56 AM
Unknown Object (File)
Sat, May 27, 12:51 AM
Unknown Object (File)
Tue, May 23, 5:30 AM
Unknown Object (File)
Fri, May 19, 8:39 PM
Unknown Object (File)
Tue, May 16, 7:47 PM
Unknown Object (File)
Mon, May 15, 9:05 PM
Unknown Object (File)
Sun, May 14, 12:01 PM
Unknown Object (File)
Tue, May 9, 5:36 PM



Unfortunately, the EcsRunLauncher doesn't support the local development
instructions becuase the local Docker containers can't use

to introspect information about itself. Additionally, we can't use the
AWS SDK to launch local docker containers via the ECS API.

This shouldn't be merged until lands
and is published to pypi - prior to that, containers will fail to stand up
because the EcsRunLauncher class won't exist yet.

Test Plan

manually tested

Diff Detail

R1 dagster
No Lint Coverage
No Test Coverage

Event Timeline


Do we have a pattern for varying our run launcher depending on the environment? Sort of like "modes" but for run launchers? Ideally, we'd use the DefaultRunLauncher when deploying locally and the EcsRunLauncher when deploying remotely.

Fixup the multi-stage target name and give daemon permission to interact with ECS. This IAM policy is a bit unrestricted at the moment - we should scope it down to just its minimum requirements as this matures.

Harbormaster returned this revision to the author for changes because remote builds failed.Jun 23 2021, 3:39 PM
Harbormaster failed remote builds in B32509: Diff 40044!
Harbormaster returned this revision to the author for changes because remote builds failed.Jun 23 2021, 5:52 PM
Harbormaster failed remote builds in B32525: Diff 40068!
dgibson added inline comments.

it would be nice to have some way of knowing before things are pushed to pypi that this example is going to break - deploy_docker has a 'from_source' folder with tests that attempt to provide that guarantee

This revision is now accepted and ready to land.Jun 24 2021, 2:18 PM

Yeah - getting this under an end-to-end integration test is something I plan to do.