HomeElementl

Configure task_definition and container_name

Description

Configure task_definition and container_name

Summary:
Previously, the EcsRunLauncher always created a new task definition.

Now, you can provide a task definition (family or ARN). If configured,
the EcsRunLauncher won't create its own task definition. This is a
coarse grained way of controlling how your runs execute. For example, if
you don't want to inheret the same memory specifications of the
parent process when running your pipeline, then you can define a new
task definition with different memory.

The EcsRunLauncher does minimal checks to ensure a task can actually be
placed using the custom task definition. It checks only that the task
definition actually exists and that there's a container that can be
overridden to execute teh dagster api execute_run command. By default,
it expects a container named "run" to be defined, but you can optionally
configure your own container name as well.

Depends on D8641

Test Plan: unit

Reviewers: dgibson, alangenfeld, johann

Reviewed By: dgibson

Differential Revision: https://dagster.phacility.com/D8642

Details

Provenance
jordansandersAuthored on Jun 30 2021, 8:26 PM
Reviewer
dgibson
Differential Revision
D8642: Configure task_definition and container_name
Parents
R1:2368e1d7da01: Ignore dagster.ExperimentalWarning
Branches
Unknown
Tags
Unknown