[feat] Support multiple aggregations and statistics at once when querying performance data#3585
Merged
vkarak merged 9 commits intoreframe-hpc:developfrom Feb 4, 2026
Merged
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## develop #3585 +/- ##
===========================================
+ Coverage 91.37% 91.39% +0.02%
===========================================
Files 62 62
Lines 13492 13520 +28
===========================================
+ Hits 12328 12357 +29
+ Misses 1164 1163 -1 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
541d9b9 to
f9f4fcc
Compare
6dfab74 to
cc60ca8
Compare
ekouts
requested changes
Jan 26, 2026
Contributor
ekouts
left a comment
There was a problem hiding this comment.
Haven't finished my testing, but stats seems to be broken for --performance-report:
./bin/reframe -c my_checks/perf_vars.py -r --performance-report "now-1d:now/stats:/"
ReFrame Setup]
version: 4.10.0-dev.0+cc60ca88
....
[==========] Finished on Mon Jan 26 11:38:45 2026+0100
WARNING: failed to generate performance report: "pval (std)" not found
Current session stored with UUID: e0a328f6-0ff3-4eeb-85fb-eb0047f17040
Log file(s) saved in '/var/folders/19/lkdv1y8949g05ynf917f07mm0000gn/T/rfm-a76sgaqq.log'
When the database is disabled, reframe doesn't complain about this, just returns the default results.
cc60ca8 to
d51db4e
Compare
Contributor
Author
vkarak
commented
Feb 1, 2026
Contributor
Author
0b20df4 to
4bc7d65
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR offers the following features:
statsaggregator. These includes mean, standard deviation, and 0.01, 0.05, 0.50, 0.95, 0.99 quantiles.mean,min,max, and also aggregations on values other than the performance, e.g.,min(num_tasks),min(pref)etc.--term-lhsand--term-rhsand are only meaningful with the--performance-compareand--performance-reportoptions.(lhs)and(rhs)respectively._Land_Rinstead of_Aand_B.Closes #3436.
Implementation details
The analytics layers is now using Polars through its Python bindings.
Managing the column names was the most challenging part of the implementation, as we wanted to keep a straightforward for selecting test attributes from the command line, specifying the left/right variants quickly and the same time produce tables with meaningful headers (e.g., include the aggregation in the column name, especially now, that multiple aggregations are supported at once). This is handled the the internal
_QueryMatchclass which replaces the very thin former_Matchnamed tuple.Todos
--table-format=prettystill uses|as a separator forjoin_uniq.Notes
If the CI does not pass for Python < 3.9, we should simply merge this PR after #3583 is addressed.This PR depends on #3596.