Page MenuHomePhabricator
Differential D5767

(buildkite 10/n) Add Windows tests
Needs RevisionPublic
Actions

Authored by nate on Thu, Dec 24, 10:24 PM.

Details

Reviewers
max
alangenfeld
rexledesma
Summary

This gets Windows builds working for a small part of core Dagster tests. While it's only a small percentage of our tests, it's a beachhead we can start with.

We'll need to do something about the test times, often 30-40 mins. From what I can tell, this typically breaks down as:

  • 5-6 mins waiting for agent - Windows hosts take a lot longer to get to ready state
  • 10-12 mins for image pull - seems that the pull is fast, but extraction/verification is very slow
  • 10-15 mins for tests to run - tests seem to be much slower on Windows

I tried swapping out the t3.large (2 vCPU burstable @ 40% baseline perf, 5Gbps network) for m5.large hosts (2 full vCPU, 10 Gbps network, 1.7x more expensive) - but didn't see a meaningful difference in the times above.

For now, I've added Windows/Windows Diffs BK pipelines that work like the integration pipelines (see test plan "windows" below).

Depends on D5895

Test Plan

windows

Diff Detail

Repository
R1 dagster
Branch
windows_tests
Lint
Lint OK
Unit
No Unit Test Coverage

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
nate updated this revision to Diff 29177.Sat, Jan 9, 12:28 AM

up

Harbormaster failed remote builds in B23994: Diff 29177!Sat, Jan 9, 1:07 AM
nate updated this revision to Diff 29198.Sat, Jan 9, 4:59 AM

up

Harbormaster failed remote builds in B24012: Diff 29198!Sat, Jan 9, 5:00 AM
nate updated this revision to Diff 29199.Sat, Jan 9, 5:07 AM

up

Harbormaster failed remote builds in B24013: Diff 29199!Sat, Jan 9, 5:08 AM
nate updated this revision to Diff 29200.Sat, Jan 9, 5:10 AM

up

Harbormaster failed remote builds in B24014: Diff 29200!Sat, Jan 9, 5:11 AM
nate updated this revision to Diff 29201.Sat, Jan 9, 5:17 AM

up

Harbormaster failed remote builds in B24015: Diff 29201!Sat, Jan 9, 5:32 AM
nate updated this revision to Diff 29203.Sat, Jan 9, 5:47 AM

up

Harbormaster failed remote builds in B24016: Diff 29203!Sat, Jan 9, 5:58 AM
nate updated this revision to Diff 29204.Sat, Jan 9, 5:58 AM

up

nate edited the summary of this revision. (Show Details)Sat, Jan 9, 6:01 AM
nate updated this revision to Diff 29205.Sat, Jan 9, 6:01 AM
nate edited the summary of this revision. (Show Details)

up

nate updated this revision to Diff 29206.Sat, Jan 9, 6:03 AM

up

Harbormaster failed remote builds in B24017: Diff 29204!Sat, Jan 9, 6:10 AM
nate updated this revision to Diff 29207.Sat, Jan 9, 6:10 AM

up

Harbormaster failed remote builds in B24020: Diff 29207!Sat, Jan 9, 6:14 AM
nate updated this revision to Diff 29208.Sat, Jan 9, 6:15 AM

up

Harbormaster failed remote builds in B24019: Diff 29206!Sat, Jan 9, 6:19 AM
Harbormaster failed remote builds in B24018: Diff 29205!
Harbormaster failed remote builds in B24021: Diff 29208!
nate updated this revision to Diff 29209.Sat, Jan 9, 6:22 AM

up

Harbormaster failed remote builds in B24022: Diff 29209!Sat, Jan 9, 6:26 AM
nate updated this revision to Diff 29210.Sat, Jan 9, 6:37 AM

up

Harbormaster failed remote builds in B24023: Diff 29210!Sat, Jan 9, 6:47 AM
nate updated this revision to Diff 29214.Sat, Jan 9, 5:09 PM

up

nate edited the summary of this revision. (Show Details)Sat, Jan 9, 5:10 PM
Harbormaster failed remote builds in B24027: Diff 29214!Sat, Jan 9, 5:20 PM
nate updated this revision to Diff 29216.Sat, Jan 9, 5:38 PM
nate edited the summary of this revision. (Show Details)

up

Harbormaster failed remote builds in B24029: Diff 29216!Sat, Jan 9, 5:56 PM
nate updated this revision to Diff 29217.Sat, Jan 9, 6:01 PM

up

Harbormaster failed remote builds in B24030: Diff 29217!Sat, Jan 9, 6:06 PM
nate updated this revision to Diff 29219.Sat, Jan 9, 6:13 PM

up

Harbormaster failed remote builds in B24032: Diff 29219!Sat, Jan 9, 6:17 PM
nate retitled this revision from (buildkite 9/n) Add Windows tests to (buildkite 10/n) Add Windows tests.Sat, Jan 9, 7:23 PM
nate edited the summary of this revision. (Show Details)
nate added a parent revision: D5895: (buildkite 9/n) Add Buildkite Windows integration image.
nate removed a parent revision: D5777: (buildkite 8/n) Make buildkite Python scripts a package.
nate updated this revision to Diff 29236.Sat, Jan 9, 10:28 PM

up

nate updated this revision to Diff 29240.Sat, Jan 9, 10:35 PM

up

Harbormaster failed remote builds in B24045: Diff 29236!Sat, Jan 9, 10:51 PM
Harbormaster failed remote builds in B24049: Diff 29240!Sat, Jan 9, 10:56 PM
nate updated this revision to Diff 29242.Sat, Jan 9, 10:57 PM

up

Harbormaster failed remote builds in B24050: Diff 29242!Sat, Jan 9, 11:01 PM
nate updated this revision to Diff 29243.Sat, Jan 9, 11:05 PM

up

Harbormaster failed remote builds in B24051: Diff 29243!Sat, Jan 9, 11:06 PM
nate updated this revision to Diff 29244.Sat, Jan 9, 11:10 PM

up

Harbormaster completed remote builds in B24052: Diff 29244.Sat, Jan 9, 11:38 PM
nate updated this revision to Diff 29245.Sat, Jan 9, 11:52 PM

up

Harbormaster failed remote builds in B24053: Diff 29245!Sun, Jan 10, 12:32 AM
nate updated this revision to Diff 29247.Sun, Jan 10, 3:34 AM

up

Harbormaster failed remote builds in B24055: Diff 29247!Sun, Jan 10, 4:05 AM
nate updated this revision to Diff 29249.Sun, Jan 10, 4:20 AM

up

Harbormaster failed remote builds in B24056: Diff 29249!Sun, Jan 10, 4:28 AM
nate updated this revision to Diff 29250.Sun, Jan 10, 4:32 AM

up

Harbormaster failed remote builds in B24057: Diff 29250!Sun, Jan 10, 5:02 AM
nate updated this revision to Diff 29254.Sun, Jan 10, 5:09 AM

up

Harbormaster completed remote builds in B24060: Diff 29254.Sun, Jan 10, 5:41 AM
nate edited the summary of this revision. (Show Details)Sun, Jan 10, 5:54 AM
nate added reviewers: max, alangenfeld.
nate added a reviewer: rexledesma.
nate updated this revision to Diff 29255.Sun, Jan 10, 5:56 AM

up

Harbormaster completed remote builds in B24061: Diff 29255.Sun, Jan 10, 6:28 AM
alangenfeld added a comment.Mon, Jan 11, 10:00 PM

woof those times are insane

python_modules/dagster/dagster_tests/cli_tests/command_tests/test_execute_command.py
136

are all these tests working on azure but failing here?

alangenfeld requested changes to this revision.Mon, Jan 11, 10:08 PM

to your queue for now - between these tests failing and the total run time im not sure this gets us in to much better of a spot than we are now on azure. Lets revisit after release

This revision now requires changes to proceed.Mon, Jan 11, 10:08 PM

Revision Contents
Changeset List

PathSize
.buildkite/
dagster-buildkite/
dagster_buildkite/
12 lines
24 lines
images/
1 line
61 lines
133 lines
steps/
19 lines
1 line
hooks/
12 lines
python_modules/
automation/
automation/
docker/
2 lines
dagster-test/
dagster/
dagster_tests/
cli_tests/
command_tests/
2 lines
17 lines
3 lines
5 lines
7 lines

Diff 29255

View Options

.buildkite/dagster-buildkite/dagster_buildkite/cli.py

Loading...
View Options

.buildkite/dagster-buildkite/dagster_buildkite/defines.py

Loading...
View Options

.buildkite/dagster-buildkite/dagster_buildkite/images/versions.py

Loading...
View Options

.buildkite/dagster-buildkite/dagster_buildkite/module_build_spec.py

Loading...
View Options

.buildkite/dagster-buildkite/dagster_buildkite/step_builder.py

Loading...
View Options

.buildkite/dagster-buildkite/dagster_buildkite/steps/windows.py

Loading...
View Options

.buildkite/dagster-buildkite/setup.py

Loading...
View Options

.buildkite/hooks/pre-checkout

Loading...
View Options

python_modules/automation/automation/docker/cli.py

Loading...
View Options

python_modules/dagster-test/dev-requirements.txt

Loading...
View Options

python_modules/dagster-test/requirements.txt

Loading...
View Options

python_modules/dagster/dagster_tests/cli_tests/command_tests/test_cli_commands.py

Loading...
View Options

python_modules/dagster/dagster_tests/cli_tests/command_tests/test_execute_command.py

Loading...
View Options

python_modules/dagster/dagster_tests/cli_tests/command_tests/test_memoized_development_cli.py

Loading...
View Options

python_modules/dagster/dagster_tests/cli_tests/command_tests/test_telemetry.py

Loading...
View Options

python_modules/dagster/tox.ini

Loading...