DPL Analysis: use offset cache for sorted grouping

[aalkin]

Original algorithm was performing extremely poorly for grouping by very large (in terms of number of rows) tables, typical for the derived data, due to being O(N^2). Precalculating the offset cache at the start solves this issue, dramatically improving the performance for the extreme cases.

@ddobrigk

[github-actions]

REQUEST FOR PRODUCTION RELEASES:
To request your PR to be included in production software, please add the corresponding labels called "async-" to your PR. Add the labels directly (if you have the permissions) or add a comment of the form (note that labels are separated by a ",")

+async-label <label1>, <label2>, !<label3> ...

This will add <label1> and <label2> and removes <label3>.

The following labels are available
async-2023-pbpb-apass4
async-2023-pp-apass4
async-2024-pp-apass1
async-2022-pp-apass7
async-2024-pp-cpass0
async-2024-PbPb-apass1
async-2024-ppRef-apass1
async-2024-PbPb-apass2
async-2023-PbPb-apass5

[ktf]

I have tested this with Correlations on hyperloop (which I understand uses grouping) and it seems to. I do not see any particular change in performance though.

@ddobrigk Could you comment on wether your tests improved? I will then merge.

[ktf]

@ddobrigk you can use my tag on hyperloop (eulisse-local) to have a build with the PR included.

[ddobrigk]

Hi @ktf yes good point. Let me test right away, will write once I have results !

[aalkin]

To clarify: the performance in the usual cases - grouping of tracks over collisions in normal AODs - should be unchanged. The performance impact will be quite noticeable in derived data that packs a lot of entries in the tables used for grouping.

[ddobrigk]

To clarify: the performance in the usual cases - grouping of tracks over collisions in normal AODs - should be unchanged. The performance impact will be quite noticeable in derived data that packs a lot of entries in the tables used for grouping.

Yes, sure. I will run the same synthetic tests in which I vary the number of elements to group from an average of 1e-2 to 100 elements per "collision", so that we can directly gauge. The main problem is I screwed something up and am now stuck recompiling (sorry). More soon ...

[ddobrigk]

@aalkin @ktf I finished running a first set of benchmarks with a 'synthetic' AO2D scenario in which I have multiple test sizes for elements being grouped to collisions, from an average of 1e-2 elements per collision to 10^4 elements per collision.

This PR helps a lot already: see the benchmark result below, with the improved version gaining, say, up to an order of magnitude in the (very common!) range around 1-10 elements (e.g. tracks, V0s, etc) grouped per grouping entity (e.g. collision): it is much improved! :-) For reference, the 'custom' grouping option corresponds to this code (very simple, very naive, and capable of dealing with the unsorted case). If you like, I can also explore other scenarios (fixed DF size at X megabytes, etc) for testing: just let me know :-)