Performance improvements based on fan-in-out example DAG

Authored by Ben Gotow <ben@foundry376.com> on Aug 22 2019, 9:49 PM.


Performance improvements based on fan-in-out example DAG

This diff makes some performance improvements to the Pipeline Explorer because I noticed performance was garbage on the example fan-in-out DAG with a 150+ solids. Here's a quick breakdown:

  • The sidebar now shows at most 20 other invocations of the solid and a "Show All" button expands the rest. This eliminates ~500ms of click latency on fan-in-out caused by 500+ invocations of same solid rendering in the sidebar every time you click a solid.
  • Eliminates another 40ms of latency (50% of remaining) when clicking to a new solid in a large dag by replacing an expensive deep comparison of props in shouldComponentUpdate.
  • Fix a bug that caused the last frame of zoom in / zoom out animations to be dropped, resulting in the SVG being drawn at 0.98812 scale, which turns out is not great for performance.
  • Remove unnecessary <G> tags, unnecessary opacity="1" attributes, and some other things that were bloating the svg.
  • Call directly through to D3's points => svg path method and memoize the results for faster drawing of the connecting lines between solids. I experimented with different ways to do this even faster, but it turns out our current approach (one <path> per connection) is actually fairly ideal because the browser culls offscreen paths when you're zoomed in and only renders the ones partially visible.
  • This one is crazy. I found that when a solid was selected and the sidebar showing solid info was *scrollable*, panning and zooming the DAG was terribly slow. I have no idea why but by poking at this for a while I found that adding position:relative to the sidebar more than doubles pan performance in this specific case where the sidebar can scroll. Might need to file a Chromium issue against this one...

Overall Result:

  • Panning is very very fast when zoomed in
  • Clicking solids is much faster
  • Zoom in / zoom out animation is slightly faster but still sorta garbage on this DAG. The browser is spending almost all the time in the Composite Layers rendering phase and I think that the size of the rasterized svg image is just enormous. We will probably need to rasterize to a canvas and transform ourselves to fix this. (Which would give us more control over the scale that we rasterize the image at, eg not 100%)

Test Plan: Run tests

Reviewers: #ft, prha

Reviewed By: #ft, prha

Subscribers: prha

Differential Revision: https://dagster.phacility.com/D869


Ben Gotow <ben@foundry376.com>Aug 22 2019, 9:51 PM
Restricted Project
Differential Revision
D869: Performance improvements based on fan-in-out example DAG
R1:a08b07d94c42: fix bad rebase