Page MenuHomePhabricator
Differential D5963

1/ Modernize airline_demo setup
ClosedPublic
Actions

Authored by max on Wed, Jan 13, 7:10 PM.

Details

Reviewers
sandyryza
yuhan
Commits
R1:1ffcc61509d8: 1/ Modernize airline_demo setup
Summary

This moves the airline demo to use the new io_manager abstractions.

Test Plan

Unit

Diff Detail

Repository
R1 dagster
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

max created this revision.Wed, Jan 13, 7:10 PM
max updated this revision to Diff 29544.Wed, Jan 13, 7:13 PM

rename

max updated this revision to Diff 29546.Wed, Jan 13, 7:14 PM

rename

Harbormaster failed remote builds in B24282: Diff 29542!Wed, Jan 13, 7:27 PM
Harbormaster failed remote builds in B24284: Diff 29544!Wed, Jan 13, 7:31 PM
Harbormaster completed remote builds in B24286: Diff 29546.
max requested review of this revision.Wed, Jan 13, 7:33 PM
max added a child revision: D5968: 2/ Modernize examples.Wed, Jan 13, 7:40 PM
sandyryza accepted this revision.EditedWed, Jan 13, 7:41 PM

Looks good! Left a comment that I think is worth addressing before merging.

examples/airline_demo/airline_demo/pipelines.py
86–87

You should be able to take out these intermediate_storage_def arguments

This revision is now accepted and ready to land.Wed, Jan 13, 7:41 PM
max marked an inline comment as done.Wed, Jan 13, 7:45 PM
max updated this revision to Diff 29556.Wed, Jan 13, 7:45 PM

comments

max updated this revision to Diff 29559.Wed, Jan 13, 7:56 PM

storage defs

Harbormaster failed remote builds in B24295: Diff 29556!Wed, Jan 13, 8:04 PM
Harbormaster completed remote builds in B24298: Diff 29559.Wed, Jan 13, 8:14 PM
Closed by commit R1:1ffcc61509d8: 1/ Modernize airline_demo setup (authored by max). · Explain WhyWed, Jan 13, 8:23 PM
This revision was automatically updated to reflect the committed changes.
max added a commit: R1:1ffcc61509d8: 1/ Modernize airline_demo setup.

Revision Contents
Changeset List

Diff 29572

View Options

examples/airline_demo/airline_demo/pipelines.py

Loading...
View Options

examples/airline_demo/airline_demo/solids.py

Loading...
View Options

examples/airline_demo/airline_demo_tests/test_pipelines.py

Loading...
View Options

examples/airline_demo/airline_demo_tests/test_solids.py

Loading...
View Options

examples/airline_demo/airline_demo_tests/test_types.py

Loading...
View Options

examples/airline_demo/airline_demo_tests/unit_tests/test_cache_file_from_s3.py

Loading...
View Options

examples/airline_demo/airline_demo_tests/unit_tests/test_ingest_csv_file_handle_to_spark.py

Loading...
View Options

examples/airline_demo/airline_demo_tests/unit_tests/test_load_data_from_spark.py

Loading...
View Options

examples/airline_demo/airline_demo_tests/unit_tests/test_unzip_file_handle.py

Loading...
View Options

examples/airline_demo/airline_demo_tests/unit_tests/testing_guide_tests/test_cache_file_from_s3_in_guide_step_four.py

Loading...
View Options

examples/airline_demo/airline_demo_tests/unit_tests/testing_guide_tests/test_cache_file_from_s3_in_guide_step_three.py

Loading...
View Options

examples/airline_demo/an_archive_member

Loading...
View Options

examples/airline_demo/setup.py

Loading...