Page MenuHomePhabricator

themissinghlink (Abhinava Singh)
User

Projects

User does not belong to any projects.

User Details

User Since
Tue, Oct 22, 4:39 PM (3 w, 11 h)

Recent Activity

Yesterday

themissinghlink added inline comments to D1396: Consolidate bay bike csv files into one csv file and upload to bucket resource.
Tue, Nov 12, 7:38 PM
themissinghlink added a comment to D1370: RFC: chain function for building strings of solids.

Pipe? a | b | c | d??? If it works for bash it works for us XD

Tue, Nov 12, 7:25 PM
themissinghlink added a comment to D1401: Add audit check for versions.

Looks good from a code perspective! Small nit

Tue, Nov 12, 6:39 PM
themissinghlink updated the diff for D1396: Consolidate bay bike csv files into one csv file and upload to bucket resource.

up

Tue, Nov 12, 6:14 PM
themissinghlink added a comment to D1370: RFC: chain function for building strings of solids.

If we support both directions, isn't the ordering clear by the direction the operators are pointing? I mean we could also keep things simple and say that you can only go in one direction IE (a >> b >> c >> d or a << b << c << d). Then the dependency ordering becomes trivial right?

Tue, Nov 12, 1:35 AM
themissinghlink updated the diff for D1396: Consolidate bay bike csv files into one csv file and upload to bucket resource.
  • added gcp support to examples
Tue, Nov 12, 1:07 AM
themissinghlink updated the summary of D1396: Consolidate bay bike csv files into one csv file and upload to bucket resource.
Tue, Nov 12, 1:00 AM
themissinghlink updated the summary of D1396: Consolidate bay bike csv files into one csv file and upload to bucket resource.
Tue, Nov 12, 12:59 AM
themissinghlink added inline comments to D1396: Consolidate bay bike csv files into one csv file and upload to bucket resource.
Tue, Nov 12, 12:57 AM
themissinghlink updated the diff for D1396: Consolidate bay bike csv files into one csv file and upload to bucket resource.
  • added tests to actually check data in bucket
  • fixed p35 compat issue with path strings
  • realized that this was all encapsulated in pipeline tests
  • WIP-just to get help from max
  • was playing around with connection pools in requests but am facing issues so reverting back
  • whoops now I reverted
  • added support for gcp bucket and went back to requests because urllib3 did not come packaged with ssl certificate issues
  • addressed review feedback
  • fixed linter bugs
  • apply arc lint fixes
Tue, Nov 12, 12:55 AM
themissinghlink added inline comments to D1396: Consolidate bay bike csv files into one csv file and upload to bucket resource.
Tue, Nov 12, 12:34 AM

Mon, Nov 11

themissinghlink updated the diff for D1396: Consolidate bay bike csv files into one csv file and upload to bucket resource.
  • WIP-just to get help from max
Mon, Nov 11, 10:32 PM
themissinghlink updated the summary of D1396: Consolidate bay bike csv files into one csv file and upload to bucket resource.
Mon, Nov 11, 5:31 PM
themissinghlink added a reviewer for D1396: Consolidate bay bike csv files into one csv file and upload to bucket resource: Restricted Project.
Mon, Nov 11, 5:01 PM

Sat, Nov 9

themissinghlink updated the diff for D1396: Consolidate bay bike csv files into one csv file and upload to bucket resource.
  • realized that this was all encapsulated in pipeline tests
Sat, Nov 9, 1:09 AM
themissinghlink updated the diff for D1396: Consolidate bay bike csv files into one csv file and upload to bucket resource.
  • fixed p35 compat issue with path strings
Sat, Nov 9, 12:59 AM

Fri, Nov 8

themissinghlink updated the diff for D1396: Consolidate bay bike csv files into one csv file and upload to bucket resource.
  • added tests to actually check data in bucket
Fri, Nov 8, 11:48 PM
themissinghlink removed a reviewer for D1396: Consolidate bay bike csv files into one csv file and upload to bucket resource: Restricted Project.
Fri, Nov 8, 11:28 PM
themissinghlink updated the summary of D1396: Consolidate bay bike csv files into one csv file and upload to bucket resource.
Fri, Nov 8, 11:26 PM
themissinghlink updated the summary of D1396: Consolidate bay bike csv files into one csv file and upload to bucket resource.
Fri, Nov 8, 11:26 PM
themissinghlink created D1396: Consolidate bay bike csv files into one csv file and upload to bucket resource.
Fri, Nov 8, 11:19 PM
themissinghlink accepted D1388: Add basic test docs.

To unblock you.

Fri, Nov 8, 7:45 PM
themissinghlink added a comment to D1388: Add basic test docs.

This is awesome. Could you also put up a preview screen shot of the docs showing the rendered view of this? I obviously trust you that this works, however, it helps with scrollback if we were to refer to this diff at any point should something go wrong and it makes my job as a reviewer easier!

Fri, Nov 8, 7:45 PM

Wed, Nov 6

themissinghlink added a comment to D1370: RFC: chain function for building strings of solids.

I had to.

Wed, Nov 6, 7:38 PM
themissinghlink created Image Macro "2chainz".
Wed, Nov 6, 7:38 PM

Tue, Nov 5

themissinghlink committed R1:014e264035e3: added scaffolding and first set of pipelines for fetching baybike data (authored by themissinghlink).
added scaffolding and first set of pipelines for fetching baybike data
Tue, Nov 5, 7:45 PM
themissinghlink closed D1350: added scaffolding and first set of pipelines for fetching baybike data.
Tue, Nov 5, 7:45 PM
themissinghlink updated the diff for D1350: added scaffolding and first set of pipelines for fetching baybike data.
  • added tests for bay bike pipelines
  • made lint fixes
  • fixed linter check that did apply
  • forgot to apply black changes
  • disabled redefined outer name rule for linting because it conflicts woth fixture pattern
  • got rid of pinned example dependencies...I get the problem now
  • forgot to make black I lost my pre commit hook
  • addressed PR feedback
Tue, Nov 5, 7:36 PM
themissinghlink added a comment to D1350: added scaffolding and first set of pipelines for fetching baybike data.

You should get all of the unziped csv files placed in a local data directory.

Is this the intended end state or just a transitory one? It seems odd to me that we want to materialize the unzipped files somewhere permanently as opposed to using a file cache or tempfile esque resource

Tue, Nov 5, 5:40 PM
themissinghlink updated the diff for D1350: added scaffolding and first set of pipelines for fetching baybike data.
  • addressed PR feedback
Tue, Nov 5, 4:11 AM
themissinghlink added a comment to D1350: added scaffolding and first set of pipelines for fetching baybike data.

@max I will add mkdir_p and use the alias in the next step (which is setting up the dagstermill stuff). The reason for this is that the next step will actually consume this pipeline so it will make a lot of sense when folks see it in the diff there!

Tue, Nov 5, 4:06 AM
themissinghlink added inline comments to D1350: added scaffolding and first set of pipelines for fetching baybike data.
Tue, Nov 5, 3:50 AM
themissinghlink added inline comments to D1354: Metaclasses.
Tue, Nov 5, 12:55 AM
themissinghlink added inline comments to D1354: Metaclasses.
Tue, Nov 5, 12:43 AM
themissinghlink updated the diff for D1350: added scaffolding and first set of pipelines for fetching baybike data.
  • forgot to make black I lost my pre commit hook
Tue, Nov 5, 12:07 AM
themissinghlink updated the diff for D1350: added scaffolding and first set of pipelines for fetching baybike data.
  • got rid of pinned example dependencies...I get the problem now
Tue, Nov 5, 12:05 AM

Mon, Nov 4

themissinghlink added reviewers for D1350: added scaffolding and first set of pipelines for fetching baybike data: Restricted Project, max.
Mon, Nov 4, 11:27 PM
themissinghlink updated the diff for D1350: added scaffolding and first set of pipelines for fetching baybike data.
  • forgot to apply black changes
Mon, Nov 4, 11:11 PM
themissinghlink updated the diff for D1350: added scaffolding and first set of pipelines for fetching baybike data.
  • fixed linter check that did apply
Mon, Nov 4, 11:04 PM
themissinghlink updated the diff for D1350: added scaffolding and first set of pipelines for fetching baybike data.
  • added tests for bay bike pipelines
  • made lint fixes
Mon, Nov 4, 11:01 PM
themissinghlink updated the summary of D1350: added scaffolding and first set of pipelines for fetching baybike data.
Mon, Nov 4, 8:12 AM
themissinghlink created D1350: added scaffolding and first set of pipelines for fetching baybike data.
Mon, Nov 4, 8:07 AM

Fri, Nov 1

themissinghlink committed R1:c362dd82e3a1: 0.6.3 (authored by themissinghlink).
0.6.3
Fri, Nov 1, 1:24 AM
themissinghlink committed R1:3dc74c84ef6b: 0.6.3.pre2 (authored by themissinghlink).
0.6.3.pre2
Fri, Nov 1, 1:04 AM
themissinghlink committed R1:dca5ad619d9b: 0.6.3.pre1 (authored by themissinghlink).
0.6.3.pre1
Fri, Nov 1, 1:01 AM
themissinghlink committed R1:8b972c5cf97f: 0.6.3.pre0 (authored by themissinghlink).
0.6.3.pre0
Fri, Nov 1, 12:54 AM
themissinghlink committed R1:e2f7616445a3: added release notes for 0.6.3 (authored by themissinghlink).
added release notes for 0.6.3
Fri, Nov 1, 12:08 AM
themissinghlink closed D1343: added release notes for 0.6.3.
Fri, Nov 1, 12:08 AM

Thu, Oct 31

themissinghlink updated the diff for D1343: added release notes for 0.6.3.
  • final polish changes
  • added documentation links
Thu, Oct 31, 11:47 PM
themissinghlink updated the diff for D1343: added release notes for 0.6.3.
  • up
Thu, Oct 31, 11:27 PM
themissinghlink updated the diff for D1343: added release notes for 0.6.3.
  • added screenshots modified messages and reordered them
Thu, Oct 31, 11:25 PM
themissinghlink created D1343: added release notes for 0.6.3.
Thu, Oct 31, 10:33 PM

Wed, Oct 30

themissinghlink added a comment to D1332: Update get_runs_with_matching_tag method on RunStorage to query for multiple tags.

I don't have much context here so everything looks legit to me (aside from the failing buildkite tests) but had a small nit that could be cool to think about! Will lean on someone else more qualified to give the approval XD

Wed, Oct 30, 8:27 PM
themissinghlink committed R1:7541080a42b9: Dag Selection Syntax (authored by themissinghlink).
Dag Selection Syntax
Wed, Oct 30, 8:12 PM
themissinghlink closed D1328: Dag Selection Syntax.
Wed, Oct 30, 8:12 PM
themissinghlink updated the diff for D1328: Dag Selection Syntax.
  • added a memoization helper to avoid the recomputation of the adjacencyMatrix and got rid of the button
  • forgot to cleanup search query logic
  • added memoize-one dependency
  • got rid of memoize-one and used weakmap also fixed a bug that caused an infinite loop because javascript doesnt evaluate empty lists to false
Wed, Oct 30, 7:18 PM
themissinghlink retitled D1328: Dag Selection Syntax from Dag Selection RFC to Dag Selection Syntax.
Wed, Oct 30, 1:53 AM
themissinghlink added inline comments to D1328: Dag Selection Syntax.
Wed, Oct 30, 1:52 AM
themissinghlink updated the diff for D1328: Dag Selection Syntax.
  • Merge branch 'master' of github.com:dagster-io/dagster into asingh-dagit-dag-selection-rfc
  • got rid of memoize-one and used weakmap also fixed a bug that caused an infinite loop because javascript doesnt evaluate empty lists to false
Wed, Oct 30, 1:47 AM

Tue, Oct 29

themissinghlink added reviewers for D1328: Dag Selection Syntax: Restricted Project, prha.
Tue, Oct 29, 8:27 PM
themissinghlink updated the diff for D1328: Dag Selection Syntax.
  • added memoize-one dependency
Tue, Oct 29, 8:26 PM
themissinghlink removed reviewers for D1328: Dag Selection Syntax: Restricted Project, prha.
Tue, Oct 29, 8:19 PM
themissinghlink updated the summary of D1328: Dag Selection Syntax.
Tue, Oct 29, 8:18 PM
themissinghlink updated the diff for D1328: Dag Selection Syntax.
  • forgot to cleanup search query logic
Tue, Oct 29, 8:12 PM
themissinghlink updated the diff for D1328: Dag Selection Syntax.
  • added a memoization helper to avoid the recomputation of the adjacencyMatrix and got rid of the button
Tue, Oct 29, 8:09 PM
themissinghlink added inline comments to D1328: Dag Selection Syntax.
Tue, Oct 29, 5:51 PM
themissinghlink added inline comments to D1328: Dag Selection Syntax.
Tue, Oct 29, 5:36 PM
themissinghlink accepted D1319: [composition] improve output node misuse errors.

Got it that makes sense! That should definitely work. Thanks for taking the time to explain!

Tue, Oct 29, 3:40 PM
themissinghlink added a comment to D1319: [composition] improve output node misuse errors.

Hmmmm...I get the use case of avoiding the accidental destructuring of tuples but I wonder if we are also imposing too much rigidity. I forsee many use cases (specifically in ML) where a common pattern is to return a tuple and have it destructured so that pieces can be passed into different parts of the control flow. Here is an example. Let's say I have a solid that downloads a dataset and throws it into a dataframe. In the simplest use case, I want to split that dataframe into a test set and a training set. These functions generally look like the following:

Tue, Oct 29, 5:17 AM
themissinghlink updated the summary of D1328: Dag Selection Syntax.
Tue, Oct 29, 2:16 AM
themissinghlink updated the summary of D1328: Dag Selection Syntax.
Tue, Oct 29, 2:13 AM
themissinghlink updated the summary of D1328: Dag Selection Syntax.
Tue, Oct 29, 2:12 AM
themissinghlink updated the diff for D1328: Dag Selection Syntax.
  • added createAdjacencyMatrix to PipelineExplorer to more efficiently extract solid dependencies and have a non working unit test which I will fix
  • added parser logic and got rid of unit test for now because this is a rfc
Tue, Oct 29, 2:01 AM

Sat, Oct 26

themissinghlink retitled D1328: Dag Selection Syntax from up to Dag Selection RFC.
Sat, Oct 26, 12:47 AM
themissinghlink updated the diff for D1328: Dag Selection Syntax.
  • added createAdjacencyMatrix to PipelineExplorer to more efficiently extract solid dependencies and have a non working unit test which I will fix
Sat, Oct 26, 12:40 AM

Fri, Oct 25

themissinghlink added a comment to D1320: Refactor validation code so its easily sharable across decorators.

Aight let's land this thing

Fri, Oct 25, 9:17 PM
themissinghlink created Image Macro "jesustakethewheel".
Fri, Oct 25, 9:15 PM
themissinghlink committed R1:c6654f41418c: Refactor validation code so its easily sharable across decorators (authored by themissinghlink).
Refactor validation code so its easily sharable across decorators
Fri, Oct 25, 9:12 PM
themissinghlink closed D1320: Refactor validation code so its easily sharable across decorators.
Fri, Oct 25, 9:12 PM
themissinghlink created D1328: Dag Selection Syntax.
Fri, Oct 25, 9:04 PM
themissinghlink updated the diff for D1320: Refactor validation code so its easily sharable across decorators.
  • got rid of docstrings in favor of check
Fri, Oct 25, 8:31 PM
themissinghlink added inline comments to D1320: Refactor validation code so its easily sharable across decorators.
Fri, Oct 25, 6:39 PM

Thu, Oct 24

themissinghlink updated the diff for D1320: Refactor validation code so its easily sharable across decorators.
  • got rid of FunctionValidationError and replaced it with InvalidFunctionInfo and cascaded it through code
Thu, Oct 24, 8:55 PM
themissinghlink added inline comments to D1320: Refactor validation code so its easily sharable across decorators.
Thu, Oct 24, 5:50 PM
themissinghlink added a reviewer for D1320: Refactor validation code so its easily sharable across decorators: Restricted Project.
Thu, Oct 24, 2:03 AM
themissinghlink updated the diff for D1320: Refactor validation code so its easily sharable across decorators.
  • revised positionals in lakehouse call
Thu, Oct 24, 1:50 AM
themissinghlink removed a reviewer for D1320: Refactor validation code so its easily sharable across decorators: Restricted Project.
Thu, Oct 24, 1:45 AM
themissinghlink updated the diff for D1320: Refactor validation code so its easily sharable across decorators.
  • forgot to make black....damnit
Thu, Oct 24, 1:32 AM
themissinghlink updated the summary of D1320: Refactor validation code so its easily sharable across decorators.
Thu, Oct 24, 1:25 AM
themissinghlink updated the diff for D1320: Refactor validation code so its easily sharable across decorators.
  • got rid of tuple checks in positional declaration
Thu, Oct 24, 1:21 AM
themissinghlink retitled D1320: Refactor validation code so its easily sharable across decorators from [part-1] refactor validation code so its easily sharable across decorators to Refactor validation code so its easily sharable across decorators.
Thu, Oct 24, 12:43 AM
themissinghlink updated the diff for D1320: Refactor validation code so its easily sharable across decorators.
  • fixed linter bugs
Thu, Oct 24, 12:30 AM
themissinghlink updated the diff for D1320: Refactor validation code so its easily sharable across decorators.
  • added positional validation and wildcard support to positional validation and added some tests
Thu, Oct 24, 12:21 AM

Wed, Oct 23

themissinghlink updated the diff for D1320: Refactor validation code so its easily sharable across decorators.
Summary: We cant import the validation functions in the types directory because there is a circular import. To avoid this and other potential import hell bugs, I broke the code out into a `decorator_utils.py` file and also added some unit tests.
- wip. Pushing up changes to branch to investigate something odd
- broke out validation functions into new file to avoid circular dependencies and also wrote some unit tests
- fixed some lint bugs whoops
Wed, Oct 23, 11:05 PM
themissinghlink created D1320: Refactor validation code so its easily sharable across decorators.
Wed, Oct 23, 6:35 PM

Tue, Oct 22

themissinghlink added a comment to D1317: added a simple string check to deserialize_json_to_dagster_namedtuple.

behold

Tue, Oct 22, 10:35 PM
themissinghlink abandoned D1318: noop change.

This was a noop change to prove to myself that my user could actually trigger jobs on buildkite.

Tue, Oct 22, 10:32 PM
themissinghlink created D1318: noop change.
Tue, Oct 22, 10:31 PM
themissinghlink updated the diff for D1317: added a simple string check to deserialize_json_to_dagster_namedtuple.

added a simple string check to deserialize_json_to_dagster_namedtuple

Tue, Oct 22, 10:07 PM
themissinghlink abandoned D1316: added a simple string check to deserialize_json_to_dagster_namedtuple.

Closing because there was a duplicate diff see D1317

Tue, Oct 22, 9:58 PM