Page MenuHomePhabricator

Add lexer/parser for subselection language
ClosedPublic

Authored by schrockn on Thu, Sep 5, 2:46 AM.

Details

Reviewers
natekupp
Group Reviewers
Restricted Project
Commits
R1:c7eefa21a1aa: Add lexer/parser for subselection language
Summary

I had a bit of time and am holding off on pressing forward on the runs db pending the dagster instance refactor.

So I made a first stab at a lexer/parser for the selection syntax to select many solids. This is largely based on
the dbt syntax. This is a bit overkill given the current language, but I strongly suspect we will make this
more complicated as time goes on, so I set it up properly.

Tracking progress in https://github.com/dagster-io/dagster/issues/1552

What this will enable (eventually) is typing at the command line or in
dagit UI. "+some_solid" which would select all solids necessary to
execute up to some_solid.

Test Plan

Buildkite

Diff Detail

Repository
R1 dagster
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

schrockn created this revision.Thu, Sep 5, 2:46 AM
schrockn updated this revision to Diff 4252.Thu, Sep 5, 3:30 AM
schrockn edited the summary of this revision. (Show Details)

up

looks reasonable to me. Main question on my end is that as this gets more sophisticated, will we want to just use an off the shelf parsing/lexing library vs building our own? I can see the logic here getting pretty complicated. Thoughts?

python_modules/dagster/dagster/core/selector/__init__.py
46

this is probably premature optimization, but this is going to do a regex compare character by character - maybe there's a cheaper way to evaluate?

natekupp accepted this revision.Thu, Sep 5, 5:28 PM
This revision is now accepted and ready to land.Thu, Sep 5, 5:28 PM

I actually think a parsing/lexing libraries are generally overrated, but then again I've never really used them that much. We've never had to use them for GraphQL and that works fine and this is way simpler, as a reference point.

python_modules/dagster/dagster/core/selector/__init__.py
46

definitely premature optimization. these are never going to be very long

This revision was automatically updated to reflect the committed changes.