Page MenuHomePhabricator
Differential D4710

Adds example dbt project.
ClosedPublic
Actions

Authored by max on Oct 7 2020, 8:51 PM.

Details

Reviewers
schrockn
bob
Commits
R1:254419a46685: Adds example dbt project.
Summary

Creates an dbt example project and Dagster pipelines to showcase dbt solids.

Test Plan

buildkite

Diff Detail

Repository
R1 dagster
Branch
arcpatch-D4710
Lint
Lint Warnings
SeverityLocationCodeMessage
Warningexamples/dbt_example/dbt_example/pipelines.py:51W0612Unused Variable
Unit
No Unit Test Coverage

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Harbormaster returned this revision to the author for changes because remote builds failed.Oct 26 2020, 4:05 PM
Harbormaster failed remote builds in B20136: Diff 24437!
max updated this revision to Diff 24449.Oct 26 2020, 5:14 PM

need to use notebooks

Harbormaster returned this revision to the author for changes because remote builds failed.Oct 26 2020, 5:36 PM
Harbormaster failed remote builds in B20146: Diff 24449!
max updated this revision to Diff 24460.Oct 26 2020, 6:08 PM

rebase on D4887

Harbormaster returned this revision to the author for changes because remote builds failed.Oct 26 2020, 6:25 PM
Harbormaster failed remote builds in B20153: Diff 24460!
max updated this revision to Diff 24479.Oct 26 2020, 7:49 PM

reflog surgery & rebase

Harbormaster returned this revision to the author for changes because remote builds failed.Oct 26 2020, 8:12 PM
Harbormaster failed remote builds in B20168: Diff 24479!
max updated this revision to Diff 24550.Oct 27 2020, 5:40 PM

up

Harbormaster returned this revision to the author for changes because remote builds failed.Oct 27 2020, 5:57 PM
Harbormaster failed remote builds in B20233: Diff 24550!
max updated this revision to Diff 24604.Oct 27 2020, 10:02 PM

rebase

Harbormaster returned this revision to the author for changes because remote builds failed.Oct 27 2020, 10:22 PM
Harbormaster failed remote builds in B20278: Diff 24604!
max updated this revision to Diff 24709.Oct 28 2020, 11:02 PM

tests

max updated this revision to Diff 24710.Oct 28 2020, 11:06 PM

lint

max updated this revision to Diff 24711.Oct 28 2020, 11:18 PM

fix installs

Harbormaster failed remote builds in B20372: Diff 24709!Oct 28 2020, 11:25 PM
Harbormaster failed remote builds in B20373: Diff 24710!Oct 28 2020, 11:37 PM
Harbormaster returned this revision to the author for changes because remote builds failed.Oct 28 2020, 11:52 PM
Harbormaster failed remote builds in B20374: Diff 24711!
max updated this revision to Diff 24744.Oct 29 2020, 4:10 PM

up

max updated this revision to Diff 24746.Oct 29 2020, 4:15 PM

rebase

max updated this revision to Diff 24748.Oct 29 2020, 4:27 PM

reqs

Harbormaster failed remote builds in B20401: Diff 24744!Oct 29 2020, 4:28 PM
max updated this revision to Diff 24749.Oct 29 2020, 4:35 PM

reqs

Harbormaster failed remote builds in B20403: Diff 24746!Oct 29 2020, 4:35 PM
max updated this revision to Diff 24750.Oct 29 2020, 4:36 PM

setup.py reqs

Harbormaster failed remote builds in B20405: Diff 24748!Oct 29 2020, 4:44 PM
max updated this revision to Diff 24752.Oct 29 2020, 4:46 PM

use port 5477

Harbormaster failed remote builds in B20406: Diff 24749!Oct 29 2020, 4:52 PM
Harbormaster failed remote builds in B20407: Diff 24750!Oct 29 2020, 4:54 PM
max updated this revision to Diff 24758.Oct 29 2020, 4:56 PM

tox

Harbormaster failed remote builds in B20409: Diff 24752!Oct 29 2020, 5:04 PM
max updated this revision to Diff 24760.Oct 29 2020, 5:06 PM

debug

max updated this revision to Diff 24762.Oct 29 2020, 5:16 PM

be explicit

Harbormaster failed remote builds in B20415: Diff 24758!Oct 29 2020, 5:19 PM
Harbormaster returned this revision to the author for changes because remote builds failed.Oct 29 2020, 5:33 PM
Harbormaster failed remote builds in B20419: Diff 24762!
Harbormaster failed remote builds in B20417: Diff 24760!
max updated this revision to Diff 24771.Oct 29 2020, 5:36 PM

debug

max updated this revision to Diff 24774.Oct 29 2020, 5:45 PM

up

max updated this revision to Diff 24776.Oct 29 2020, 5:54 PM

debug

Harbormaster returned this revision to the author for changes because remote builds failed.Oct 29 2020, 5:59 PM
Harbormaster failed remote builds in B20428: Diff 24771!
Harbormaster failed remote builds in B20433: Diff 24776!
Harbormaster failed remote builds in B20431: Diff 24774!Oct 29 2020, 6:07 PM
max updated this revision to Diff 24785.Oct 29 2020, 6:28 PM

conn

Harbormaster returned this revision to the author for changes because remote builds failed.Oct 29 2020, 6:35 PM
Harbormaster failed remote builds in B20442: Diff 24785!
max updated this revision to Diff 24791.Oct 29 2020, 6:37 PM

up

Harbormaster returned this revision to the author for changes because remote builds failed.Oct 29 2020, 6:47 PM
Harbormaster failed remote builds in B20446: Diff 24791!
max updated this revision to Diff 24794.Oct 29 2020, 6:50 PM

setup

Harbormaster returned this revision to the author for changes because remote builds failed.Oct 29 2020, 6:56 PM
Harbormaster failed remote builds in B20449: Diff 24794!
max updated this revision to Diff 24795.Oct 29 2020, 6:57 PM

fix relative path

max updated this revision to Diff 24798.Oct 29 2020, 7:03 PM

fix coverage

Harbormaster failed remote builds in B20450: Diff 24795!Oct 29 2020, 7:03 PM
Harbormaster completed remote builds in B20452: Diff 24798.Oct 29 2020, 7:10 PM
max requested review of this revision.Oct 29 2020, 7:10 PM
max updated this revision to Diff 24804.Oct 29 2020, 7:14 PM

undebug

Harbormaster completed remote builds in B20457: Diff 24804.Oct 29 2020, 7:31 PM
bob accepted this revision.Oct 29 2020, 7:42 PM

Macro coolbeans:

This is super cool (and well tested)! integrating dbt -> jupyter plotting -> slack is a much more comprehensive showcase of dagster. thanks again for commandeering this

examples/dbt_example/dbt_example/solids.py
9 ↗(On Diff #24804)

Not blocking, but this constant doesn't seem to be used anywhere atm

Once this diff lands, the URL could be changed to a GitHub raw content on the dagster repo rather than a gist. Then, a configured version of download_file would allow the pipeline to run without needing solid config

This revision is now accepted and ready to land.Oct 29 2020, 7:42 PM
Closed by commit R1:254419a46685: Adds example dbt project. (authored by bob, committed by max). · Explain WhyOct 29 2020, 7:44 PM
This revision was automatically updated to reflect the committed changes.
max added a commit: R1:254419a46685: Adds example dbt project..

Revision Contents
Changeset List

PathSize
examples/
dbt_example/
1 line
62 lines
dbt_example/
8 lines
51 lines
dbt_example_project/
8 lines
19 lines
analysis/
data/
78 lines
38 lines
macros/
1 line
models/
cereals/
3 lines
19 lines
101 lines
3 lines
3 lines
3 lines
12 lines
snapshots/
tests/
19 lines
5 lines
21 lines
17 lines
24 lines

Diff 23982

View Options

examples/dbt_example/MANIFEST.in

Loading...
View Options

examples/dbt_example/README.md

Loading...
View Options

examples/dbt_example/dbt_example/__init__.py

Loading...
View Options

examples/dbt_example/dbt_example/pipelines.py

Loading...
View Options

examples/dbt_example/dbt_example_project/.gitignore

Loading...
View Options

examples/dbt_example/dbt_example_project/README.md

Loading...
View Options

examples/dbt_example/dbt_example_project/analysis/.gitkeep

Loading...
View Options

examples/dbt_example/dbt_example_project/data/.gitkeep

Loading...
View Options

examples/dbt_example/dbt_example_project/data/cereals.csv

Loading...
View Options

examples/dbt_example/dbt_example_project/dbt_project.yml

Loading...
View Options

examples/dbt_example/dbt_example_project/macros/.gitkeep

Loading...
View Options

examples/dbt_example/dbt_example_project/macros/log_macro.sql

Loading...
View Options

examples/dbt_example/dbt_example_project/models/cereals/least_caloric.sql

Loading...
View Options

examples/dbt_example/dbt_example_project/models/cereals/normalize_cereals.sql

Loading...
View Options

examples/dbt_example/dbt_example_project/models/cereals/schema.yml

Loading...
View Options

examples/dbt_example/dbt_example_project/models/cereals/sort_by_calories.sql

Loading...
View Options

examples/dbt_example/dbt_example_project/models/cereals/sort_cold_cereals_by_calories.sql

Loading...
View Options

examples/dbt_example/dbt_example_project/models/cereals/sort_hot_cereals_by_calories.sql

Loading...
View Options

examples/dbt_example/dbt_example_project/profiles.yml

Loading...
View Options

examples/dbt_example/dbt_example_project/snapshots/.gitkeep

Loading...
View Options

examples/dbt_example/dbt_example_project/tests/.gitkeep

Loading...
View Options

examples/dbt_example/docker-compose.yml

Loading...
View Options

examples/dbt_example/requirements.txt

Loading...
View Options

examples/dbt_example/setup.py

Loading...
View Options

examples/dbt_example/setup.sh

Loading...
View Options

examples/dbt_example/tox.ini

Loading...