Page MenuHomeElementl

RFC solid hook takes event_iterator
AbandonedPublic

Authored by yuhan on Apr 20 2021, 6:09 AM.

Details

Reviewers
None
Summary

this diff change the generic hook (not a public api) arg event_list to event_iterator
it uses itertools.tee to make copies of the original event iterators. there will be n+1 copies where n=len(all_hook_defs) bc iterators are stateful and each hook wants to keep its own independent state. this approach may result in high mem cost when n is large (https://docs.python.org/3/library/itertools.html#itertools.tee)

Test Plan

unit

Diff Detail

Repository
R1 dagster
Branch
yuhan/hook-iter
Lint
Lint Passed
Unit
No Test Coverage