Page MenuHomeElementl

1/ load versioned mdx files from s3 instead of local .versioned_content folder
ClosedPublic

Authored by yuhan on Jun 16 2021, 6:14 PM.

Details

Summary

when building static pages, instead of reading from .versioned_content folder, this diff loads versioned content from s3.

next steps:

  • versioned images
  • update the release pipeline (or update version script) to upload versioned content to s3 instead of checking in to the repo
  • remove versioned_content and versioned_images
  • update release to skip rc version
Test Plan

preview

Diff Detail

Repository
R1 dagster
Branch
yuhan/docs-versioned-content
Lint
Lint Passed
Unit
No Test Coverage

Event Timeline

Harbormaster returned this revision to the author for changes because remote builds failed.Jun 16 2021, 6:57 PM
Harbormaster failed remote builds in B32209: Diff 39675!
Harbormaster returned this revision to the author for changes because remote builds failed.Jun 17 2021, 10:00 PM
Harbormaster failed remote builds in B32340: Diff 39831!
Harbormaster returned this revision to the author for changes because remote builds failed.Jun 17 2021, 10:29 PM
Harbormaster failed remote builds in B32346: Diff 39838!
jordansanders added inline comments.
docs/next/pages/[...page].tsx
301

If we don't pass the credentials argument here, it should read them from wherever the credentials provider hits. When running on buildkite (on ec2), that'll be the instance profile. When running locally, that'll either be an envvar or an ~/.aws/credentials file (depending on how people have it set up).

https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/setting-credentials-node.html

In general, I think using instance profiles is a better strategy for providing credentials because it removes the risk or leaking secrets when set as envvars.

when remote content isn't available, fall back to the local content (i.e. master version)

yuhan requested review of this revision.Jun 26 2021, 2:41 AM

make s3 bucket public readonly - content serving website should just be public readble

yuhan retitled this revision from load versioned mdx files from s3 instead of local .versioned_content folder to 1/ load versioned mdx files from s3 instead of local .versioned_content folder.Jun 28 2021, 6:43 AM
This revision is now accepted and ready to land.Jun 29 2021, 3:55 PM