Page MenuHomeElementl

Stub AWS before instance construction
ClosedPublic

Authored by jordansanders on Jun 30 2021, 8:43 PM.
Tags
None
Referenced Files
Unknown Object (File)
Fri, Nov 25, 7:33 AM
Unknown Object (File)
Fri, Nov 18, 12:26 AM
Unknown Object (File)
Fri, Nov 18, 12:25 AM
Unknown Object (File)
Fri, Nov 18, 12:25 AM
Unknown Object (File)
Fri, Nov 18, 12:25 AM
Unknown Object (File)
Thu, Nov 17, 10:01 AM
Unknown Object (File)
Thu, Nov 17, 8:55 AM
Unknown Object (File)
Tue, Nov 15, 12:25 AM
Subscribers
None

Details

Summary

Previously, we were stubbing our AWS objects (the ECS resource and EC2
client) after an instance was constructed. This was simpler, but it
prevented us from using our stubbed objects in the launcher's
__init__ function.

Instead, we're now stubbing our AWS objects before the launcher has bee
initialized. Once the fixture is called, any call to boto3.client()
(with any arguments) will return our ecs fixture. Any call to
boto3.resource() (with any arguments) will return our ec2 fixture.
Should we ever make multiple calls to either boto3.client() or
boto3.resource(), we'll need to modify this fixture.

Now, we'll be able to use our stubbed objects inside the launcher's
__init__ - for example, if we wanted to do an invariant check to make
sure something like a cluster or task definition already exists.

Depends on D8638

Test Plan

unit

Diff Detail

Repository
R1 dagster
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

dgibson added inline comments.
python_modules/libraries/dagster-aws/dagster_aws_tests/ecs_tests/test_launcher.py
84–96

as discussedin the other diff this pattern does not cleanup correctly, but we can sort that out on that diff

This revision is now accepted and ready to land.Jul 1 2021, 2:31 PM

Rearrange stack so this comes first