Diffusion dagster 0bfc6665cd76

create dagster-aws-pyspark library with emr_pyspark_step_launcher
0bfc6665cd76
Actions

Description

create dagster-aws-pyspark library with emr_pyspark_step_launcher

Test Plan: bk, pip install

Reviewers: schrockn, nate

Reviewed By: schrockn

Differential Revision: https://dagster.phacility.com/D4806

Details

Provenance

sandyryza

Authored on Oct 16 2020, 8:52 PM

Reviewer

schrockn

Differential Revision

D4806: create dagster-aws-pyspark library with emr_pyspark_step_launcher

Parents

R1:b9da214e9619: Adds CSS styling for doc pages generated by sphinx-click.

Branches

Unknown

Tags

Unknown

Reverted By

R1:696fd43cb149: Revert dagster-aws-pyspark
D4940: Revert dagster-aws-pyspark

Event Timeline

sandyryza committed R1:0bfc6665cd76: create dagster-aws-pyspark library with emr_pyspark_step_launcher (authored by sandyryza).Oct 26 2020, 3:14 PM

sandyryza added an edge: D4806: create dagster-aws-pyspark library with emr_pyspark_step_launcher.

sandyryza added a reverting change: D4940: Revert dagster-aws-pyspark.Oct 28 2020, 4:48 PM

sandyryza added a reverting change: R1:696fd43cb149: Revert dagster-aws-pyspark.Oct 29 2020, 6:32 PM

Changes (36)

Path

Size

.buildkite/

pipeline.py

Makefile

docs-requirements.txt

docs/

next/

src/

pages/

overview/

packages/

libraries.mdx

sections/

api/

apidocs/

libraries/

dagster_aws.rst

dagster_aws_pyspark.rst

examples/

airline_demo/

emr_pyspark/

legacy_examples/

dagster_examples/

simple_pyspark/

pipelines.py

setup.py

tox.ini

python_modules/

libraries/

dagster_aws_pyspark_tests

test_version.py

dev-requirements.txt

	dagster-aws-pyspark/
	dagster-aws/

	dagster_aws_pyspark/
	dagster_aws/	emr/

pyspark_step_launcher.py

	dagster_aws_pyspark_tests/	test_pyspark_step_launcher.py/
	dagster_aws_tests/	emr_tests/


	test_pyspark.py

tox.ini

dagster-aws/

dagster_aws/

emr/

__init__.py

pyspark_step_launcher.py

dagster_aws_tests/

emr_tests/

test_pyspark.py

test_pyspark_step_launcher.py

setup.py

	libraries/	dagster-aws-pyspark/
		dagster-test/

setup.py

	python_modules/

	libraries/
	examples/

	dagster-aws-pyspark/
	airline_demo/

.coveragerc

	dagster-aws-pyspark/
	airflow_ingest/

	requirements.txt
	__init__.py

	libraries/	dagster-aws-pyspark/	dagster_aws_pyspark_tests/
		examples/	airflow_ingest/

__init__.py

R1:0bfc6665cd76

View Options

.buildkite/pipeline.py

View Options

Makefile

View Options

docs-requirements.txt

View Options

docs/next/src/pages/overview/packages/libraries.mdx

View Options

docs/sections/api/apidocs/libraries/dagster_aws.rst

View Options

docs/sections/api/apidocs/libraries/dagster_aws_pyspark.rst

View Options

examples/airline_demo/airline_demo/pipelines.py

View Options

examples/airline_demo/setup.py

View Options

examples/airline_demo/tox.ini

View Options

examples/emr_pyspark/repo.py

View Options

examples/emr_pyspark/requirements.txt

View Options

examples/emr_pyspark/tox.ini

View Options

examples/legacy_examples/Dockerfile

View Options

examples/legacy_examples/dagster_examples/simple_pyspark/pipelines.py

View Options

examples/legacy_examples/setup.py

View Options

examples/legacy_examples/tox.ini

View Options

python_modules/libraries/dagster-aws-pyspark/.coveragerc

View Options

python_modules/libraries/dagster-aws-pyspark/README.md

View Options

python_modules/libraries/dagster-aws-pyspark/dagster_aws_pyspark/init.py

View Options

python_modules/libraries/dagster-aws-pyspark/dagster_aws_pyspark/pyspark_step_launcher.py

View Options

python_modules/libraries/dagster-aws-pyspark/dagster_aws_pyspark/version.py

View Options

python_modules/libraries/dagster-aws-pyspark/dagster_aws_pyspark_tests/init.py

View Options

python_modules/libraries/dagster-aws-pyspark/dagster_aws_pyspark_tests/test_pyspark_step_launcher.py

View Options

python_modules/libraries/dagster-aws-pyspark/dagster_aws_pyspark_tests/test_version.py

View Options

python_modules/libraries/dagster-aws-pyspark/dev-requirements.txt

View Options

python_modules/libraries/dagster-aws-pyspark/requirements.txt

View Options

python_modules/libraries/dagster-aws-pyspark/setup.py

View Options

python_modules/libraries/dagster-aws-pyspark/tox.ini

View Options

python_modules/libraries/dagster-aws/dagster_aws/emr/init.py

View Options

python_modules/libraries/dagster-aws/dagster_aws/emr/pyspark_step_launcher.py

View Options

python_modules/libraries/dagster-aws/dagster_aws_tests/emr_tests/test_pyspark.py

View Options

python_modules/libraries/dagster-aws/dagster_aws_tests/emr_tests/test_pyspark_step_launcher.py

View Options

python_modules/libraries/dagster-aws/setup.py

create dagster-aws-pyspark library with emr_pyspark_step_launcher0bfc6665cd76Actions

Description

Details

Event Timeline

Changes (36)

R1:0bfc6665cd76

.buildkite/pipeline.py

Makefile

docs-requirements.txt

docs/next/src/pages/overview/packages/libraries.mdx

docs/sections/api/apidocs/libraries/dagster_aws.rst

docs/sections/api/apidocs/libraries/dagster_aws_pyspark.rst

examples/airline_demo/airline_demo/pipelines.py

examples/airline_demo/setup.py

examples/airline_demo/tox.ini

examples/emr_pyspark/repo.py

examples/emr_pyspark/requirements.txt

examples/emr_pyspark/tox.ini

examples/legacy_examples/Dockerfile

examples/legacy_examples/dagster_examples/simple_pyspark/pipelines.py

examples/legacy_examples/setup.py

examples/legacy_examples/tox.ini

python_modules/libraries/dagster-aws-pyspark/.coveragerc

python_modules/libraries/dagster-aws-pyspark/README.md

python_modules/libraries/dagster-aws-pyspark/dagster_aws_pyspark/__init__.py

python_modules/libraries/dagster-aws-pyspark/dagster_aws_pyspark/pyspark_step_launcher.py

python_modules/libraries/dagster-aws-pyspark/dagster_aws_pyspark/version.py

python_modules/libraries/dagster-aws-pyspark/dagster_aws_pyspark_tests/__init__.py

python_modules/libraries/dagster-aws-pyspark/dagster_aws_pyspark_tests/test_pyspark_step_launcher.py

python_modules/libraries/dagster-aws-pyspark/dagster_aws_pyspark_tests/test_version.py

python_modules/libraries/dagster-aws-pyspark/dev-requirements.txt

python_modules/libraries/dagster-aws-pyspark/requirements.txt

python_modules/libraries/dagster-aws-pyspark/setup.py

python_modules/libraries/dagster-aws-pyspark/tox.ini

python_modules/libraries/dagster-aws/dagster_aws/emr/__init__.py

python_modules/libraries/dagster-aws/dagster_aws/emr/pyspark_step_launcher.py

python_modules/libraries/dagster-aws/dagster_aws_tests/emr_tests/test_pyspark.py

python_modules/libraries/dagster-aws/dagster_aws_tests/emr_tests/test_pyspark_step_launcher.py

python_modules/libraries/dagster-aws/setup.py

create dagster-aws-pyspark library with emr_pyspark_step_launcher
0bfc6665cd76
Actions

python_modules/libraries/dagster-aws-pyspark/dagster_aws_pyspark/init.py

python_modules/libraries/dagster-aws-pyspark/dagster_aws_pyspark_tests/init.py

python_modules/libraries/dagster-aws/dagster_aws/emr/init.py