Differential D8684 Diff 40976 docs/content/deployment/guides/airflow.mdx

Changeset View

Standalone View

docs/content/deployment/guides/airflow.mdx

	Show First 20 Lines • Show All 267 Lines • ▼ Show 20 Lines
	By default, the `DagsterDockerOperator` will bind-mount `/tmp` on the host into `/tmp` in the Docker container. You can control this by setting the `op_kwargs` in `make_airflow_dag`. For instance, if you'd prefer to mount `/host_tmp` on the host into `/container_tmp` in the container, and use this volume for intermediate storage, you can run:			By default, the `DagsterDockerOperator` will bind-mount `/tmp` on the host into `/tmp` in the Docker container. You can control this by setting the `op_kwargs` in `make_airflow_dag`. For instance, if you'd prefer to mount `/host_tmp` on the host into `/container_tmp` in the container, and use this volume for intermediate storage, you can run:

	```python file=/deploying/airflow/mounted.py			```python file=/deploying/airflow/mounted.py
	from dagster_airflow.factory import make_airflow_dag			from dagster_airflow.factory import make_airflow_dag

	dag, steps = make_airflow_dag(			dag, steps = make_airflow_dag(
	module_name="docs_snippets.intro_tutorial.airflow",			module_name="docs_snippets.intro_tutorial.airflow",
	pipeline_name="hello_cereal_pipeline",			pipeline_name="hello_cereal_pipeline",
	run_config={"storage": {"filesystem": {"config": {"base_dir": "/container_tmp"}}}},			run_config={
				"storage": {"filesystem": {"config": {"base_dir": "/container_tmp"}}}
				},
	dag_id=None,			dag_id=None,
	dag_description=None,			dag_description=None,
	dag_kwargs=None,			dag_kwargs=None,
	op_kwargs={"host_tmp_dir": "/host_tmp", "tmp_dir": "/container_tmp"},			op_kwargs={"host_tmp_dir": "/host_tmp", "tmp_dir": "/container_tmp"},
	)			)
	```			```

	## Compatibility			## Compatibility

	Note that Airflow versions less than 1.10.3 are incompatible with Python 3.7+.			Note that Airflow versions less than 1.10.3 are incompatible with Python 3.7+.