Page MenuHomeElementl

Increase gRPC server max_worker count to default
ClosedPublic

Authored by sashank on Feb 4 2021, 4:12 PM.

Details

Summary

We previously set the default max_worker count to 1 for the gRPC server. By setting this value to None, the ThreadPoolExecutor uses min(32, (os.cpu_count() or 1) + 4) max workers by default.

Test Plan

These methods and classes are already under test

Diff Detail

Repository
R1 dagster
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

  • Remove condition on heartbeat and always check invariant
Harbormaster returned this revision to the author for changes because remote builds failed.Feb 4 2021, 4:30 PM
Harbormaster failed remote builds in B25288: Diff 30822!
Harbormaster returned this revision to the author for changes because remote builds failed.Feb 4 2021, 5:05 PM
Harbormaster failed remote builds in B25291: Diff 30825!

recommend just adding CHANGES.md entry in the diff

python_modules/dagster/dagster/grpc/server.py
1021โ€“1022

maybe just say "use the grpc default" since it may change? "

This revision is now accepted and ready to land.Feb 4 2021, 5:53 PM
python_modules/dagster/dagster/grpc/server.py
1021โ€“1022

good call

This revision was landed with ongoing or failed builds.Feb 4 2021, 6:01 PM
This revision was automatically updated to reflect the committed changes.