Page MenuHomeElementl

Configure task_definition and container_name
ClosedPublic

Authored by jordansanders on Jun 30 2021, 8:48 PM.
Tags
None
Referenced Files
Unknown Object (File)
Mon, Nov 28, 6:32 PM
Unknown Object (File)
Thu, Nov 24, 10:41 AM
Unknown Object (File)
Sat, Nov 19, 5:43 AM
Unknown Object (File)
Fri, Nov 18, 12:26 AM
Unknown Object (File)
Fri, Nov 18, 12:26 AM
Unknown Object (File)
Fri, Nov 18, 12:26 AM
Unknown Object (File)
Fri, Nov 18, 12:26 AM
Unknown Object (File)
Fri, Nov 18, 12:26 AM
Subscribers
None

Details

Summary

Previously, the EcsRunLauncher always created a new task definition.

Now, you can provide a task definition (family or ARN). If configured,
the EcsRunLauncher won't create its own task definition. This is a
coarse grained way of controlling how your runs execute. For example, if
you don't want to inheret the same memory specifications of the
parent process when running your pipeline, then you can define a new
task definition with different memory.

The EcsRunLauncher does minimal checks to ensure a task can actually be
placed using the custom task definition. It checks only that the task
definition actually exists and that there's a container that can be
overridden to execute teh dagster api execute_run command. By default,
it expects a container named "run" to be defined, but you can optionally
configure your own container name as well.

Depends on D8641

Test Plan

unit

Diff Detail

Repository
R1 dagster
Lint
Lint Not Applicable
Unit
Tests Not Applicable