HomePhabricator

[Dask] Put Dask execution in IEngine interface

Authored by Nate Kupp <nate@elementl.com> on Jul 16 2019, 9:51 PM.

Description

[Dask] Put Dask execution in IEngine interface

Summary:
This diff breaks up execute_on_dask to split out a DaskEngine object; it also uses the executor config to construct the engine, so the core can invoke the DaskEngine without needing to know about Dask via the executor_config property on RunConfig

I'll use this in a follow-up diff which will drop execute_on_dask entirely in favor of execute_pipeline w/ RunConfig.

Test Plan: unit

Reviewers: #ft, max

Reviewed By: #ft, max

Subscribers: max

Differential Revision: https://dagster.phacility.com/D637

Details

Committed
Nate Kupp <nate@elementl.com>Jul 16 2019, 10:15 PM
Reviewer
Restricted Project
Differential Revision
D637: [Dask] Put Dask execution in IEngine interface
Parents
R1:6494edbfd525: Fix grqphql playground asset paths
Branches
Unknown
Tags
Unknown