Major Update to DashPVA

[Osayi-ANL]

PR: Feature - Post Analysis Workbench and 3D Visualization Suite

Overview

This PR introduces the new Post Analysis Workbench, a significant expansion of the data inspection capabilities. It allows users to transition between 1D, 2D, and 3D perspectives of scanned data. Additionally, it provides a specialized 3D Analysis Suite for volumetric slicing and internal data inspection.

Key Features

• Post Analysis Workbench: Added a unified interface to toggle between:
  • 1D: Line profiles and ROI integrated intensity over time/steps.
  • 2D: Enhanced heatmap rendering for detector images.
  • 3D: Full volumetric rendering of reciprocal space or image stacks.
• Slicing Tools: Integrated orthographic slicing within the 3D toolset to allow users to cut through data volumes for detailed structural analysis.
• HDF5 Data Alignment: Updated the HDF5Writer to output data in the standardized /entry/data/metadata/HKL/ hierarchy, ensuring full compatibility with the Post Analysis Workbench loading mechanism.

Structural Changes (HDF5)

To support these tools, the saving mechanism now explicitly includes:

• NAME Datasets: Axis groups now contain human-readable labels (e.g., Mu, Eta, Phi).
• UB Matrix Normalization: UB matrices are now consistently saved as 9-element flat arrays.
• RSM Mapping: q-space coordinates (qx, qy, qz) are correctly mapped to the 3D viewing volume.

Notes for Reviewers

• Verify that the NAME labels are appearing correctly in the UI dropdowns for axis selection.
• Test the 3D slice performance on larger H5 datasets.
• Ensure outputs/temp/temp.h5 is correctly overwritten during live scans.

References

• See CHANGELOG.md for version details.

[pecomyint]

• hdf5plugin missing from environment.yml. (add that)

• utils/hdf5_writer.py (line 55) hardcodes Path('~/hdf5/').expanduser(). This might not be desired for all users or deployment environments.

• database/db.py uses sqlite:///dashpva.db. This creates the DB in the current working directory, which can vary depending on how the app is launched. (good to put that in back to DashPV)

• utils/hdf5_loader.py writes to hdf5_loader_errors.txt and error_output.txt in the current working directory (good to put that in back to DashPV)

• inconsistent logging - utils/metadata_converter.py uses the logging module and writes to a logs/ directory.
  utils/hdf5_loader.py manually opens and writes to text files (error_output.txt).

• utils/hdf5_writer.py: The save_caches_to_h5 method contains duplicated logic for saving data. The code block for saving (lines 96-122) is nearly identical to the code block for the uncompressed fallback (lines 125-165) (simply the codes and avoid duplications!)

• utils/hdf5_loader.py: The methods load_h5_to_2d and load_h5_images_2d contain only pass. If these are intended to be implemented later, they should raise NotImplementedError or log a warning rather than silently doing nothing. (Add warming or raise error please)