Uses grpcio-health-check to add the health checking protocol to the grpc server. Then adds a client that can be invoked in the dagster cli.
TODO: add a failing test
Integrations, manual helm install and check that liveness probe is active
dagster api grpc --port 3030 & dagster api grpc-health-check --port 3030 # $? == 0 dagster api grpc-health-check --port 3031 # $? == 1
The only alternative I know is to have the probe be in the main container. The advantage of this approach is that users don't need to remember to include the probe binary in their images
Is there anything we can do to make it easier for users to include/remember to include the binary in their user deployment images? It could just be bundled in dagster and invoked via a dagster cli but I don't think that's great, especially given that the binary is only available for linux. We could build our own client for it?
Actually restartPolicy isn't possible either https://github.com/kubernetes/kubernetes/issues/24725, guess it has to be packaged with the container.
looks fine - you could add a server.wait() after thew with() block ends, but it should still shut down on its own without it