Page MenuHomeElementl

Stub ECS RegisterTaskDefinition
ClosedPublic

Authored by jordansanders on Jun 16 2021, 4:01 PM.

Details

Summary

Testing options for ECS are limited:

  • Moto's mocking of ECS is incomplete
  • Localstack's ECS implementation is part of their Pro offering and it's not immediately clear that it'd be sufficient
  • Integration testing against a live endpoint is slower, requires additional infrastructure management, and require real credentials in Buildkite. Also, many of the ECS operations are immutable. For example, once we register a task definition, we can never delete it - we can only mark it as inactive.

Instead, I've opted to stub the ECS endpoints that we'll need for an
EcsRunLauncher beginning with RegisterTaskDefinition:

https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_RegisterTaskDefinition.html

To add some additional confidence to the stubs, I'm using botocore's
Stubber:

https://botocore.amazonaws.com/v1/documentation/api/latest/reference/stubber.html

Stubber validates both parameters and responses - it won't allow you to
stub an invalid response structure and it won't allow you to pass
invalid parameters. However, some internal behaviors of ECS still need
to be faked. For example, when registering a task definition to the same
family, ECS automatically increments the revision number. In our
StubbedEcs, this is achieved by storing task definitions in an in memory
dictionary.

While subject to all the usual downsides of stubbing, this approach
gives me enough confidence to meaningfully unit test against ECS. It
doesn't prevent us from eventually adding a handful of end-to-end tests
against a live ECS endpoint.

Test Plan

unit

Diff Detail

Repository
R1 dagster
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

Harbormaster returned this revision to the author for changes because remote builds failed.Jun 16 2021, 4:26 PM
Harbormaster failed remote builds in B32162: Diff 39622!
Harbormaster returned this revision to the author for changes because remote builds failed.Jun 16 2021, 6:04 PM
Harbormaster failed remote builds in B32193: Diff 39659!

Move stubber into its own function to resolve mypy complaints

Harbormaster returned this revision to the author for changes because remote builds failed.Jun 16 2021, 8:24 PM
Harbormaster failed remote builds in B32230: Diff 39698!
dgibson added inline comments.
python_modules/libraries/dagster-aws/dagster_aws_tests/ecs_tests/stubbed_ecs.py
82

yikes

This revision is now accepted and ready to land.Jun 17 2021, 10:10 PM
This revision was automatically updated to reflect the committed changes.