perf: SIMD-accelerated hex encoding/decoding and image operations

[JSKitty]

Summary

This PR introduces comprehensive SIMD acceleration for performance-critical operations in Vector's backend, targeting both ARM64 (Apple Silicon, Android) and x86_64 (Windows, Linux) platforms.

Changes

1. Hex Encoding (bytes → hex string)

Implementation:

• ARM64 (NEON): Uses vqtbl1q_u8 (TBL instruction) for 16-byte parallel lookup table operations
• x86_64 (AVX2): Processes all 32 bytes in a single operation using 256-bit registers with _mm256_blendv_epi8 for conditional ASCII conversion
• x86_64 (SSE2): Fallback using 128-bit registers, processes 16 bytes per iteration

Algorithm: Split bytes into nibbles, compare > 9 to identify hex letters, add appropriate ASCII offset ('0' for digits, 'a'-10 for letters), interleave and store.

Benchmarks (32 bytes → 64 hex chars):

Method	Time	Speedup
format!("{:x}")	~1630 ns	baseline
Scalar LUT	~35 ns	47x
NEON (ARM64)	~26 ns	62x
AVX2 (x86_64)	~25 ns	65x

2. Hex Decoding (hex string → bytes)

Implementation:

• ARM64 (NEON): Optimized algorithm using simplified nibble conversion: (char & 0x0F) + 9*(char has bit 0x40 set)
  • For '0'-'9': (0x30-0x39 & 0x0F) = 0-9, bit 0x40 not set → +0
  • For 'A'-'F'/'a'-'f': (0x41-0x46 & 0x0F) = 1-6, bit 0x40 set → +9 = 10-15
  • Uses vsliq_n_u8 (SLI - Shift Left and Insert) to combine nibbles in one instruction
  • Uses vuzp1q_u8/vuzp2q_u8 for deinterleaving
• x86_64 (SSE2): Uses comparison-based digit/letter detection with _mm_cmplt_epi8

Benchmarks (64 hex chars → 32 bytes):

Method	Time	Cycles	Speedup
Scalar LUT	~19 ns	~61	baseline
SSE2 (x86_64)	~5 ns	~16	3.8x
NEON (ARM64)	~2.5 ns	~8	7.6x

3. Alpha Transparency Check

Implementation:

• Processes 128 bytes (32 RGBA pixels) per iteration
• ANDs all chunks together - if any alpha < 255, result shows it
• Checks alpha bytes at positions 3, 7, 11, 15 (every 4th byte)
• Parallel processing with rayon for images > 4MB (256KB chunks for L2 cache efficiency)

Benchmarks (27 MP image, 109 MB RGBA):

Method	Time	Speedup
Scalar	5.37 ms	baseline
SIMD + Parallel	0.59 ms	9.1x

Theoretical minimum at 200 GB/s memory bandwidth: 0.55ms

4. Set Alpha Opaque

Implementation:

• ORs alpha mask (0xFF at positions 3,7,11,15) with pixel data
• Same parallelization strategy as alpha check

Benchmarks (27 MP image):

Method	Time	Speedup
Scalar	3.08 ms	baseline
SIMD + Parallel	0.67 ms	4.6x

5. RGB → RGBA Conversion

Implementation:

• ARM64 (NEON): Uses vld3q_u8 to load RGB data deinterleaved into R/G/B planes, then vst4q_u8 to store as RGBA with alpha=255
• x86_64 (SSSE3): Uses _mm_shuffle_epi8 (pshufb) to rearrange 12 RGB bytes → 16 RGBA bytes per iteration

Performance: ~4x speedup on large images compared to naive scalar

Cross-Platform Compatibility

• Runtime feature detection: AVX2/SSSE3 detected at runtime with is_x86_feature_detected!
• Proper #[target_feature] annotations: All SIMD functions properly annotated
• Endian-safe scalar fallbacks: Uses #[cfg(target_endian = "little/big")] for conditional compilation
• Overflow protection: All size calculations use checked_mul() to prevent overflow on large inputs

Stability Fixes

• Fixed out-of-bounds SIMD reads in rgb_to_rgba_ssse3 (16-byte loads require proper bounds: i + 52 for 16-pixel loop, i + 16 for 4-pixel loop)
• Added input validation to nearest_neighbor_downsample
• Added #[target_feature(enable = "sse2")] to SSE2 functions for proper code generation

Testing

• All existing tests pass
• Roundtrip tests verify encode/decode correctness
• Large input tests exercise SIMD paths
• Uppercase hex handling verified

Files Changed

• src/simd/hex.rs - Hex encoding/decoding (947 lines)
• src/simd/image.rs - Image operations (937 lines)

[github-actions]

Claude finished @JSKitty's task in 1m 29s —— View job

---

PR Review: SIMD-accelerated hex encoding/decoding and image operations

• Analyze PR context and changes
• Review SIMD hex encoding/decoding implementation (src/simd/hex.rs)
• Review SIMD image operations (src/simd/image.rs)
• Review SIMD module structure (src/simd/mod.rs)
• Review integration changes across the codebase
• Compile findings and provide feedback

---

Summary

This is a well-structured PR introducing SIMD acceleration for performance-critical operations. The implementation demonstrates solid understanding of SIMD intrinsics across ARM64 (NEON) and x86_64 (SSE2/AVX2) platforms. The code is well-documented with clear performance benchmarks.

Positive Aspects

1. Excellent documentation: The module-level docs and inline comments explaining algorithms are exemplary
2. Proper target feature annotations: #[target_feature(enable = "...")] is correctly used on all SIMD functions
3. Runtime feature detection: AVX2 is properly detected at runtime on x86_64 with SSE2 fallback
4. Overflow protection: Size calculations use checked_mul() to prevent integer overflow
5. Comprehensive tests: Good coverage of roundtrip tests and edge cases
6. Smart parallelization: 4MB threshold and 256KB chunk size for L2 cache efficiency is well-reasoned

Issues and Suggestions

Medium Priority

1. Missing #[target_feature] on bytes_to_hex_16 for x86_64

File: src-tauri/src/simd/hex.rs:286-324

The bytes_to_hex_16 function for x86_64 uses SSE2 intrinsics but lacks the #[target_feature(enable = "sse2")] attribute. While SSE2 is baseline for x86_64, adding the attribute ensures proper code generation and is consistent with other functions.

#[cfg(target_arch = "x86_64")]
#[target_feature(enable = "sse2")]  // Add this
#[inline]
pub fn bytes_to_hex_16(bytes: &[u8; 16]) -> String {

Fix this →

2. Inefficient hex_to_bytes_16 implementation on x86_64

File: src-tauri/src/simd/hex.rs:735-749

The x86_64 implementation of hex_to_bytes_16 creates unnecessary allocations:

let full = unsafe { hex_decode_32_sse2(&[h, &[b'0'; 32]].concat()) };

This allocates a new 64-byte Vec on every call just to reuse the 32-byte decoder. Consider adding a dedicated hex_decode_16_sse2 function or use a stack-allocated array with padding.

3. Potential UB: Invalid hex input handling silently corrupts data

File: src-tauri/src/simd/hex.rs:496-497

The comment states "Invalid hex characters are treated as 0x00" but this could lead to silent data corruption. Consider adding validation or returning Result<[u8; 32], Error> instead.

Low Priority

4. Unused alpha mask bytes in set_alpha functions

File: src-tauri/src/simd/image.rs:395-398

The AVX2 alpha mask byte order looks reversed. In _mm256_set_epi8, bytes are specified in reverse order (high to low), but the pattern (-1, 0, 0, 0, -1, 0, 0, 0, ...) puts alpha at positions 0,4,8,12,16,20,24,28 instead of 3,7,11,15,19,23,27,31. However, since RGBA has alpha at position 3 within each 4-byte pixel, and _mm256_set_epi8 fills in reverse, this actually works correctly. Just wanted to flag this as worth double-checking with a test on actual image data.

5. Consider using MaybeUninit for uninitialized buffers

File: src-tauri/src/simd/image.rs:602-603

The nearest_neighbor_downsample function uses set_len on a Vec after allocation. While this works, using MaybeUninit would be more idiomatic and safer:

let mut result: Vec<MaybeUninit<u8>> = Vec::with_capacity(dst_size);

6. Scalar fallback in bytes_to_hex_string duplicates HEX_CHARS constant

File: src-tauri/src/simd/hex.rs:456

HEX_CHARS is defined inline in the x86_64 path. Consider reusing HEX_NIBBLE or extracting to a shared constant.

Style Suggestions

1. The doc comment in nearest_neighbor_downsample (src/simd/image.rs:579-591) duplicates documentation - consider consolidating
2. Some functions could benefit from #[must_use] attribute (e.g., bytes_to_hex_32, hex_to_bytes_32)

Security Considerations

• No obvious security issues identified
• The hash calculations for file integrity use the SIMD hex encoding properly
• Crypto operations in crypto.rs correctly use the new hex utilities

Testing Recommendations

The existing tests are good, but consider adding:

1. Fuzz tests for hex encoding/decoding edge cases
2. Tests with exactly 16 bytes of input for hex_to_bytes_16 on x86_64 (to cover the inefficient allocation path)
3. Property-based tests verifying decode(encode(x)) == x for random inputs

Verdict

Overall: Approve with minor suggestions ✅

This is solid, performance-focused work with proper safety considerations. The issues noted are minor and don't block merging. The performance improvements are substantial (up to 65x for hex encoding, 9x for image operations) and the code quality is high.

---

[YuurinBee]

ACK-LGTM