Page MenuHomePhabricator

themissinghlink (Abhinava Singh)
User

Projects

User does not belong to any projects.

User Details

User Since
Oct 22 2019, 4:39 PM (13 w, 2 d)

Recent Activity

Today

themissinghlink added a comment to D1874: Bay Bikes Weather Pipeline refactor.

ready for a once over! Good call on the naming change and the container switch, this actually makes my general life a lot easier as well since I switched my dagster-home to point to the test-postgres-db.

Thu, Jan 23, 9:31 PM
themissinghlink updated the diff for D1874: Bay Bikes Weather Pipeline refactor.
  • renamed dagster_components to common and switched to using the test-postgres-db container
Thu, Jan 23, 9:17 PM
themissinghlink added reviewers for D1874: Bay Bikes Weather Pipeline refactor: prha, sashank, alangenfeld.
Thu, Jan 23, 6:43 AM
themissinghlink updated the diff for D1874: Bay Bikes Weather Pipeline refactor.
  • tweaked test because c/i has no api key installed
Thu, Jan 23, 6:38 AM
themissinghlink removed reviewers for D1874: Bay Bikes Weather Pipeline refactor: prha, alangenfeld, sashank.
Thu, Jan 23, 6:35 AM
themissinghlink updated the summary of D1874: Bay Bikes Weather Pipeline refactor.
Thu, Jan 23, 6:35 AM
themissinghlink created D1874: Bay Bikes Weather Pipeline refactor.
Thu, Jan 23, 6:27 AM
themissinghlink abandoned D1833: Add schedules to weather pipelines.

Realized that if we wanted to switch to the partition API, we need to design my pipelines to require transactional consistency. cloud storage is not that, so we need to switch to using a database. I should have done this a long time ago, but am going to do a bunch of incremental refactors to get there instead of blowing up this diff.

Thu, Jan 23, 1:11 AM
themissinghlink committed R1:81745d294c4c: BUGFIX: Ensure SubprocessExecutionManager isn't leaking POSIX semaphores (authored by themissinghlink).
BUGFIX: Ensure SubprocessExecutionManager isn't leaking POSIX semaphores
Thu, Jan 23, 12:42 AM
themissinghlink closed D1857: BUGFIX: Ensure SubprocessExecutionManager isn't leaking POSIX semaphores.
Thu, Jan 23, 12:42 AM
themissinghlink updated the diff for D1857: BUGFIX: Ensure SubprocessExecutionManager isn't leaking POSIX semaphores.

Repushing to bring in buildkite nbconvert issues.

Thu, Jan 23, 12:31 AM
themissinghlink accepted D1872: Only do python3 support for pipenv smoke.

LGTM

Thu, Jan 23, 12:23 AM
themissinghlink updated the diff for D1857: BUGFIX: Ensure SubprocessExecutionManager isn't leaking POSIX semaphores.

up

Thu, Jan 23, 12:11 AM

Yesterday

themissinghlink updated the diff for D1857: BUGFIX: Ensure SubprocessExecutionManager isn't leaking POSIX semaphores.

up

Wed, Jan 22, 11:22 PM
themissinghlink updated the diff for D1857: BUGFIX: Ensure SubprocessExecutionManager isn't leaking POSIX semaphores.
  • oops added comment, ready to land
Wed, Jan 22, 11:15 PM
themissinghlink updated the diff for D1857: BUGFIX: Ensure SubprocessExecutionManager isn't leaking POSIX semaphores.
  • switched to using seven for abc
Wed, Jan 22, 11:13 PM
themissinghlink updated the diff for D1857: BUGFIX: Ensure SubprocessExecutionManager isn't leaking POSIX semaphores.
  • switched to using iterables
Wed, Jan 22, 11:05 PM
themissinghlink updated the diff for D1857: BUGFIX: Ensure SubprocessExecutionManager isn't leaking POSIX semaphores.
  • got rid of mock since we tweaked the test
Wed, Jan 22, 10:21 PM
themissinghlink updated the diff for D1857: BUGFIX: Ensure SubprocessExecutionManager isn't leaking POSIX semaphores.
  • tweaked tests to not reach into internals of subprocess manager
Wed, Jan 22, 10:20 PM
themissinghlink added inline comments to D1857: BUGFIX: Ensure SubprocessExecutionManager isn't leaking POSIX semaphores.
Wed, Jan 22, 8:10 PM
themissinghlink added inline comments to D1857: BUGFIX: Ensure SubprocessExecutionManager isn't leaking POSIX semaphores.
Wed, Jan 22, 8:08 PM
themissinghlink added a comment to D1865: Fix example schedules.

You would probably need to run the schedule from a test and watch it fail right?

Wed, Jan 22, 6:47 PM
themissinghlink accepted D1865: Fix example schedules.

LGTM

Wed, Jan 22, 6:45 PM
themissinghlink updated the diff for D1857: BUGFIX: Ensure SubprocessExecutionManager isn't leaking POSIX semaphores.
  • fixed tests to not totally suck and also fixed up some inconsistencies I noticed in other tests
Wed, Jan 22, 6:31 PM

Sat, Jan 18

themissinghlink added a comment to D1857: BUGFIX: Ensure SubprocessExecutionManager isn't leaking POSIX semaphores.

Yeah you’re right. I was reading but wasn’t sure how to isolate a pipeline run, pipeline, Dagster instance, etc. which I need to do in order to make sure lsof can be inspected. As supposed to mocking, How do we do that in other tests?

Sat, Jan 18, 1:30 AM

Fri, Jan 17

themissinghlink added a reviewer for D1857: BUGFIX: Ensure SubprocessExecutionManager isn't leaking POSIX semaphores: alangenfeld.
Fri, Jan 17, 11:03 PM
themissinghlink updated the diff for D1857: BUGFIX: Ensure SubprocessExecutionManager isn't leaking POSIX semaphores.
  • oops lint fix
Fri, Jan 17, 10:59 PM
themissinghlink updated the diff for D1857: BUGFIX: Ensure SubprocessExecutionManager isn't leaking POSIX semaphores.
  • added mocking testing abilities dep
Fri, Jan 17, 10:58 PM
themissinghlink retitled D1857: BUGFIX: Ensure SubprocessExecutionManager isn't leaking POSIX semaphores from added fix and tests to BUGFIX: Ensure SubprocessExecutionManager isn't leaking POSIX semaphores.
Fri, Jan 17, 10:55 PM
themissinghlink created D1857: BUGFIX: Ensure SubprocessExecutionManager isn't leaking POSIX semaphores.
Fri, Jan 17, 10:48 PM

Wed, Jan 15

themissinghlink committed R1:e73a5930770f: Add github resource (#2054) (authored by Joseph Van Drunen <josephvandrunen@gmail.com>).
Add github resource (#2054)
Wed, Jan 15, 11:49 PM

Tue, Jan 14

themissinghlink added reviewers for D1833: Add schedules to weather pipelines: alangenfeld, prha, max.
Tue, Jan 14, 12:17 AM

Mon, Jan 13

themissinghlink updated the diff for D1833: Add schedules to weather pipelines.
  • made bay bike examples mypy compatible
Mon, Jan 13, 9:56 PM
themissinghlink retitled D1833: Add schedules to weather pipelines from clean up weather pipelines and make it work for retraining to Add schedules to weather pipelines.
Mon, Jan 13, 9:22 PM
themissinghlink created D1833: Add schedules to weather pipelines.
Mon, Jan 13, 9:16 PM

Sat, Jan 11

themissinghlink accepted D1820: fix partition decorators to respect mode.

LGTM I had a small nit but you can take it or leave it.

Sat, Jan 11, 4:48 AM

Fri, Jan 10

themissinghlink added a comment to D1775: Dagster Pandas Guide Docs.

@schrockn Just added more changes since last time, will need a once over whenever ur free!

Fri, Jan 10, 10:24 PM
themissinghlink updated the diff for D1775: Dagster Pandas Guide Docs.
  • added documentation for dataframe level validation
Fri, Jan 10, 2:17 AM
themissinghlink updated the diff for D1775: Dagster Pandas Guide Docs.
  • made copy edits to documentation to ensure it reads better. Thanks prha
  • made inline fixes
  • edit docs based on nicks feedback
Fri, Jan 10, 1:41 AM
themissinghlink updated the summary of D1775: Dagster Pandas Guide Docs.
Fri, Jan 10, 1:28 AM
themissinghlink updated the summary of D1775: Dagster Pandas Guide Docs.
Fri, Jan 10, 1:27 AM
themissinghlink updated the summary of D1775: Dagster Pandas Guide Docs.
Fri, Jan 10, 1:20 AM
themissinghlink updated the summary of D1775: Dagster Pandas Guide Docs.
Fri, Jan 10, 1:19 AM
themissinghlink updated the summary of D1775: Dagster Pandas Guide Docs.
Fri, Jan 10, 1:18 AM
themissinghlink updated the diff for D1775: Dagster Pandas Guide Docs.
  • made copy edits to documentation to ensure it reads better. Thanks prha
  • made inline fixes
  • edit docs based on nicks feedback
Fri, Jan 10, 1:15 AM

Thu, Jan 9

themissinghlink committed R1:766a61ebc5b6: Add Shape Constraint Support (authored by themissinghlink).
Add Shape Constraint Support
Thu, Jan 9, 7:04 PM
themissinghlink closed D1778: Add Shape Constraint Support.
Thu, Jan 9, 7:04 PM

Tue, Jan 7

themissinghlink added a comment to D1778: Add Shape Constraint Support.

Great suggestions. The factory copy is def more clear now. Ready for a once over!

Tue, Jan 7, 11:20 PM
themissinghlink added a reviewer for D1778: Add Shape Constraint Support: schrockn.
Tue, Jan 7, 11:19 PM
themissinghlink updated the summary of D1778: Add Shape Constraint Support.
Tue, Jan 7, 11:13 PM
themissinghlink updated the summary of D1778: Add Shape Constraint Support.
Tue, Jan 7, 11:12 PM
themissinghlink updated the diff for D1778: Add Shape Constraint Support.
  • fixed bay bike tests to adapt to new api changes
Tue, Jan 7, 11:10 PM
themissinghlink removed reviewers for D1778: Add Shape Constraint Support: alangenfeld, max, schrockn.
Tue, Jan 7, 11:03 PM
themissinghlink updated the diff for D1778: Add Shape Constraint Support.
  • addressed feedback and made factory params names more generic
  • made lint fixes
Tue, Jan 7, 10:59 PM

Mon, Jan 6

themissinghlink added a reviewer for D1775: Dagster Pandas Guide Docs: schrockn.
Mon, Jan 6, 10:45 PM
themissinghlink edited reviewers for D1778: Add Shape Constraint Support, added: schrockn; removed: prha.
Mon, Jan 6, 10:42 PM
themissinghlink updated the summary of D1778: Add Shape Constraint Support.
Mon, Jan 6, 8:40 PM
themissinghlink added a comment to D1778: Add Shape Constraint Support.

Added @max because he might find this interesting!

Mon, Jan 6, 7:20 PM
themissinghlink added a reviewer for D1778: Add Shape Constraint Support: max.
Mon, Jan 6, 7:19 PM
themissinghlink updated the summary of D1778: Add Shape Constraint Support.
Mon, Jan 6, 6:44 PM

Sat, Jan 4

themissinghlink added reviewers for D1778: Add Shape Constraint Support: alangenfeld, prha.
Sat, Jan 4, 4:06 AM
themissinghlink updated the summary of D1778: Add Shape Constraint Support.
Sat, Jan 4, 4:01 AM
themissinghlink updated the diff for D1778: Add Shape Constraint Support.
  • fixed pylint issue
Sat, Jan 4, 3:51 AM
themissinghlink updated the diff for D1778: Add Shape Constraint Support.
  • make pytest p2 compatible
Sat, Jan 4, 3:47 AM
themissinghlink updated the diff for D1778: Add Shape Constraint Support.
  • pylint fix
Sat, Jan 4, 3:45 AM
themissinghlink updated the summary of D1778: Add Shape Constraint Support.
Sat, Jan 4, 3:39 AM
themissinghlink created D1778: Add Shape Constraint Support.
Sat, Jan 4, 3:38 AM

Fri, Jan 3

themissinghlink updated the diff for D1775: Dagster Pandas Guide Docs.
  • made copy edits to documentation to ensure it reads better. Thanks prha
Fri, Jan 3, 11:15 PM
themissinghlink added reviewers for D1775: Dagster Pandas Guide Docs: alangenfeld, max, prha.
Fri, Jan 3, 10:57 PM
themissinghlink updated the summary of D1775: Dagster Pandas Guide Docs.
Fri, Jan 3, 10:51 PM
themissinghlink updated the summary of D1775: Dagster Pandas Guide Docs.
Fri, Jan 3, 10:51 PM
themissinghlink updated the summary of D1775: Dagster Pandas Guide Docs.
Fri, Jan 3, 10:50 PM
themissinghlink created D1775: Dagster Pandas Guide Docs.
Fri, Jan 3, 10:50 PM
themissinghlink accepted D1774: Escape html entities to pass lint check.

LGTM

Fri, Jan 3, 9:30 PM

Thu, Jan 2

themissinghlink committed R1:83d8a6c9fd64: add markdown descriptions for dagster pandas dataframe types (authored by themissinghlink).
add markdown descriptions for dagster pandas dataframe types
Thu, Jan 2, 7:01 PM
themissinghlink closed D1753: add markdown descriptions for dagster pandas dataframe types.
Thu, Jan 2, 7:01 PM
themissinghlink added inline comments to D1753: add markdown descriptions for dagster pandas dataframe types.
Thu, Jan 2, 6:51 PM
themissinghlink updated the diff for D1753: add markdown descriptions for dagster pandas dataframe types.
  • fixed tests
Thu, Jan 2, 6:50 PM
themissinghlink updated the diff for D1753: add markdown descriptions for dagster pandas dataframe types.
  • pushed up black fix
  • tweaked to make markdown more compact
Thu, Jan 2, 6:39 PM
themissinghlink added inline comments to D1753: add markdown descriptions for dagster pandas dataframe types.
Thu, Jan 2, 6:37 PM
themissinghlink added a comment to D1753: add markdown descriptions for dagster pandas dataframe types.

How about this. This is starting to look a lot more concise, you were totally right.

Thu, Jan 2, 6:25 PM
themissinghlink added a comment to D1753: add markdown descriptions for dagster pandas dataframe types.

Was actually playing around with that just now, Hold my beer

Thu, Jan 2, 6:03 PM
themissinghlink added a comment to D1753: add markdown descriptions for dagster pandas dataframe types.

We could fix the line padding to make the spaces between things smaller, but the padding is sadly controlled via css I believe and the above is a result of me using the fewest possible new lines without totally borking the structure.

Thu, Jan 2, 5:58 PM
themissinghlink added a comment to D1753: add markdown descriptions for dagster pandas dataframe types.

@alangenfeld, how does this look:

Thu, Jan 2, 5:52 PM

Mon, Dec 30

themissinghlink added reviewers for D1753: add markdown descriptions for dagster pandas dataframe types: alangenfeld, max, schrockn.
Mon, Dec 30, 7:16 PM
themissinghlink updated the diff for D1753: add markdown descriptions for dagster pandas dataframe types.
  • pushed up black fix
Mon, Dec 30, 7:04 PM
themissinghlink updated the summary of D1753: add markdown descriptions for dagster pandas dataframe types.
Mon, Dec 30, 7:03 PM
themissinghlink updated the summary of D1753: add markdown descriptions for dagster pandas dataframe types.
Mon, Dec 30, 7:02 PM
themissinghlink created D1753: add markdown descriptions for dagster pandas dataframe types.
Mon, Dec 30, 6:56 PM

Dec 19 2019

themissinghlink accepted D1718: Eliminate overridable type_check in custom data frame.

Provisional LGTM once you get tests passing.

Dec 19 2019, 11:14 PM
themissinghlink added inline comments to D1718: Eliminate overridable type_check in custom data frame.
Dec 19 2019, 11:13 PM
themissinghlink added inline comments to D1718: Eliminate overridable type_check in custom data frame.
Dec 19 2019, 11:11 PM
themissinghlink added inline comments to D1718: Eliminate overridable type_check in custom data frame.
Dec 19 2019, 10:40 PM
themissinghlink committed R1:4924d306bac3: migrate all dataframes to new dagster pandas api (authored by themissinghlink).
migrate all dataframes to new dagster pandas api
Dec 19 2019, 9:54 PM
themissinghlink closed D1699: migrate all dataframes to new dagster pandas api.
Dec 19 2019, 9:54 PM
themissinghlink updated the diff for D1699: migrate all dataframes to new dagster pandas api.
  • reverted changes to use the original dagster pandas api
Dec 19 2019, 9:20 PM
themissinghlink added inline comments to D1718: Eliminate overridable type_check in custom data frame.
Dec 19 2019, 9:18 PM
themissinghlink added inline comments to D1715: Convert custom pandas df types to vanilla instances of RuntimeType.
Dec 19 2019, 7:49 PM
themissinghlink added inline comments to D1715: Convert custom pandas df types to vanilla instances of RuntimeType.
Dec 19 2019, 7:49 PM
themissinghlink accepted D1715: Convert custom pandas df types to vanilla instances of RuntimeType.

This is great! I have thought about it and I think going the route of not being mypy compliant and getting people to use input/output definitions is the better move because it makes it clear that Dagster Runtime objects are NOT extensions of python types, they are a different abstraction altogether.

Dec 19 2019, 7:47 PM