decouple api docs dependencies from make dev_install
This diff decouple the docs requirements from the dev_install.
So, for example, dagster-airflow api docs can be built properly (it requires both dagster-airflow and airlfow but we can't install airflow in dev_install as discussed in D6206)
We will use pip install -r docs-requirements.txt instead of make dev_install to install docs dependency
I updated the "Publish the docs" section too to reflect the change.
In the long term, the plan with the new site is to use a docker image to release the docs so it won't be affected by individual local virtualenv.
in a fresh virtualenv
~/dev/dagster yuhan/docs-auto-build* $ test-docs-auto-build $ pip install -r docs-requirements.txt -qqq astroid 2.4.2 has requirement lazy-object-proxy==1.4.*, but you'll have lazy-object-proxy 1.5.2 which is incompatible. tox 3.14.2 has requirement importlib-metadata<2,>=1.1.0; python_version < "3.8", but you'll have importlib-metadata 3.4.0 which is incompatible. marshmallow-sqlalchemy 0.24.1 has requirement marshmallow>=3.0.0, but you'll have marshmallow 2.21.0 which is incompatible. flask-jwt-extended 3.25.0 has requirement PyJWT<2.0,>=1.6.4, but you'll have pyjwt 2.0.1 which is incompatible. apache-airflow 1.10.10 has requirement attrs~=19.3, but you'll have attrs 20.3.0 which is incompatible. apache-airflow 1.10.10 has requirement croniter<0.4,>=0.3.17, but you'll have croniter 1.0.6 which is incompatible. apache-airflow 1.10.10 has requirement jinja2<2.11.0,>=2.10.1, but you'll have jinja2 2.11.3 which is incompatible. apache-airflow 1.10.10 has requirement pandas<1.0.0,>=0.17.1, but you'll have pandas 1.2.1 which is incompatible. apache-airflow 1.10.10 has requirement werkzeug<1.0.0, but you'll have werkzeug 1.0.1 which is incompatible. botocore 1.12.253 has requirement docutils<0.16,>=0.10, but you'll have docutils 0.16 which is incompatible. botocore 1.12.253 has requirement urllib3<1.26,>=1.20; python_version >= "3.4", but you'll have urllib3 1.26.3 which is incompatible. databricks-cli 0.12.2 has requirement tenacity>=6.2.0, but you'll have tenacity 4.12.0 which is incompatible. dagster-pandas 0.10.2 has requirement pandas<=1.1.4, but you'll have pandas 1.2.1 which is incompatible. great-expectations 0.12.10 has requirement requests<2.24,>=2.20, but you'll have requests 2.25.1 which is incompatible. snowflake-connector-python 2.3.10 has requirement pytz<2021.0, but you'll have pytz 2021.1 which is incompatible. twilio 6.51.1 has requirement PyJWT==1.7.1, but you'll have pyjwt 2.0.1 which is incompatible.
although there are many conflicts, it doesn't affect the Sphinx build process. We still get all related api docs generated correctly - i think it's because those incompatible packages weren't used when generating the api references.
make buildnext && cd next && yarn && yarn dev: localhost has no empty api page
Reviewers: #docs, nate
Reviewed By: nate
Differential Revision: https://dagster.phacility.com/D6246