Page MenuHomeElementl

specify partition for asset events
ClosedPublic

Authored by prha on Dec 23 2020, 1:20 AM.

Diff Detail

Repository
R1 dagster
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

prha created this revision.

update schema

prha requested review of this revision.Dec 23 2020, 1:40 AM

Will defer to @sandyryza for Python review but this looks good! Excited to get this in!

Nice

python_modules/dagster/dagster/core/storage/event_log/sql_event_log.py
569

Do you happen to know whether this will properly use the index you defined in the migration diff?

This revision is now accepted and ready to land.Jan 4 2021, 11:17 PM
python_modules/dagster/dagster/core/storage/event_log/sql_event_log.py
569

I didn't add an index on partition.

I can modify https://dagster.phacility.com/D5760 to add an index on partition, but yes, this would use the index if it existed.

python_modules/dagster/dagster/core/storage/event_log/sql_event_log.py
569

If I understand correctly, you added an index on (asset_key, partition)?

I'm referring to this line in https://dagster.phacility.com/D5760:

op.create_index(
    "idx_asset_partition", "event_logs", ["asset_key", "partition"], unique=False
)
python_modules/dagster/dagster/core/storage/event_log/sql_event_log.py
569

ah, that's pretty good.

I was looking at the schema file, which doesn't create the index... I'll add that there.

This revision was automatically updated to reflect the committed changes.