Page MenuHomeElementl

(buildkite 10/n) Add Windows tests
AbandonedPublic

Authored by nate on Dec 24 2020, 10:24 PM.

Details

Summary

This gets Windows builds working for a small part of core Dagster tests. While it's only a small percentage of our tests, it's a beachhead we can start with.

We'll need to do something about the test times, often 30-40 mins. From what I can tell, this typically breaks down as:

  • 5-6 mins waiting for agent - Windows hosts take a lot longer to get to ready state
  • 10-12 mins for image pull - seems that the pull is fast, but extraction/verification is very slow
  • 10-15 mins for tests to run - tests seem to be much slower on Windows

I tried swapping out the t3.large (2 vCPU burstable @ 40% baseline perf, 5Gbps network) for m5.large hosts (2 full vCPU, 10 Gbps network, 1.7x more expensive) - but didn't see a meaningful difference in the times above.

For now, I've added Windows/Windows Diffs BK pipelines that work like the integration pipelines (see test plan "windows" below).

Depends on D5895

Test Plan

windows

Diff Detail

Repository
R1 dagster
Branch
windows_tests
Lint
Lint Passed
Unit
No Test Coverage

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
nate retitled this revision from (buildkite 9/n) Add Windows tests to (buildkite 10/n) Add Windows tests.Jan 9 2021, 7:23 PM
nate edited the summary of this revision. (Show Details)
nate added reviewers: max, alangenfeld.
nate added a reviewer: rexledesma.

woof those times are insane

python_modules/dagster/dagster_tests/cli_tests/command_tests/test_execute_command.py
136

are all these tests working on azure but failing here?

to your queue for now - between these tests failing and the total run time im not sure this gets us in to much better of a spot than we are now on azure. Lets revisit after release

This revision now requires changes to proceed.Jan 11 2021, 10:08 PM