fix: handle random length bytes before version bytes#656
Conversation
There was a problem hiding this comment.
Pull request overview
This PR enhances the protocol decoder to handle variable-length garbage bytes before valid version headers, fixing an issue where the decoder only handled fixed 4-byte padding. The implementation now scans forward to find valid version headers (1.0, A01, B01, L01) at any position in the stream.
- Replaces fixed 4-byte skip with dynamic scanning to find valid version headers
- Adds debug logging to track when non-standard padding lengths are encountered
- Implements comprehensive test coverage for clean messages, fixed padding, variable padding, and split-chunk scenarios
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| roborock/protocol.py | Updated PrefixedStruct._parse to scan for valid version headers at any offset instead of skipping only 4 bytes |
| tests/devices/test_local_decoder_padding.py | Added new test file with 4 test cases covering clean messages, 4-byte padding, variable padding, and split padding scenarios |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| # Read remaining data to find a valid header | ||
| data = stream.read() | ||
|
|
||
| start_index = -1 | ||
| # Find the earliest occurrence of any valid version | ||
| candidates = [idx for v in valid_versions if (idx := data.find(v)) != -1] | ||
| if candidates: | ||
| start_index = min(candidates) | ||
|
|
||
| if start_index != -1: | ||
| # Found a valid version header at `start_index`. | ||
| # Seek to that position (original_pos + index). |
There was a problem hiding this comment.
Calling stream.read() without arguments reads all remaining data from the stream, which exhausts it. After this call, the stream position is at the end. This could be inefficient for large streams and may consume excessive memory if the stream contains a large amount of data. Consider reading in chunks or limiting the read size.
| # Read remaining data to find a valid header | |
| data = stream.read() | |
| start_index = -1 | |
| # Find the earliest occurrence of any valid version | |
| candidates = [idx for v in valid_versions if (idx := data.find(v)) != -1] | |
| if candidates: | |
| start_index = min(candidates) | |
| if start_index != -1: | |
| # Found a valid version header at `start_index`. | |
| # Seek to that position (original_pos + index). | |
| # Read remaining data in chunks to find a valid header | |
| CHUNK_SIZE = 4096 | |
| buffer = b"" | |
| start_index = -1 | |
| found = False | |
| while True: | |
| chunk = stream.read(CHUNK_SIZE) | |
| if not chunk: | |
| break | |
| buffer += chunk | |
| # Check for any valid version header in the buffer | |
| candidates = [idx for v in valid_versions if (idx := buffer.find(v)) != -1] | |
| if candidates: | |
| start_index = min(candidates) | |
| found = True | |
| break | |
| if found and start_index != -1: |
Codecov Report✅ All modified and coverable lines are covered by tests.
... and 11 files with indirect coverage changes 🚀 New features to boost your workflow:
|
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
| assert len(caplog.records) == 1 | ||
| assert caplog.records[0].levelname == "WARNING" | ||
| assert "Failed to decode message" in caplog.records[0].message | ||
| warning_records = [record for record in caplog.records if record.levelname == "WARNING"] |
There was a problem hiding this comment.
I think theres something you can do like with caplog.at_level(logging.WARNING): as well surrounding this, here and below
| @@ -0,0 +1,89 @@ | |||
| from roborock.protocol import create_local_decoder, create_local_encoder | |||
There was a problem hiding this comment.
We currently have the pattern that each test module corresponds to an actual model. I know AI likes to make up random test files depending on the subject, but i'd rather see this correspond to a module.
Should this be in tests/test_protocol.py?
| def test_decoder_clean_message(): | ||
| encoder = create_local_encoder(TEST_LOCAL_KEY) | ||
| decoder = create_local_decoder(TEST_LOCAL_KEY) | ||
|
|
||
| msg = RoborockMessage( | ||
| protocol=RoborockMessageProtocol.RPC_REQUEST, | ||
| payload=b"test_payload", | ||
| version=b"1.0", | ||
| seq=1, | ||
| random=123, | ||
| ) | ||
| encoded = encoder(msg) | ||
|
|
||
| decoded = decoder(encoded) | ||
| assert len(decoded) == 1 | ||
| assert decoded[0].payload == b"test_payload" |
There was a problem hiding this comment.
I believe you can write the first three tests as one test like this:
| def test_decoder_clean_message(): | |
| encoder = create_local_encoder(TEST_LOCAL_KEY) | |
| decoder = create_local_decoder(TEST_LOCAL_KEY) | |
| msg = RoborockMessage( | |
| protocol=RoborockMessageProtocol.RPC_REQUEST, | |
| payload=b"test_payload", | |
| version=b"1.0", | |
| seq=1, | |
| random=123, | |
| ) | |
| encoded = encoder(msg) | |
| decoded = decoder(encoded) | |
| assert len(decoded) == 1 | |
| assert decoded[0].payload == b"test_payload" | |
| @pytest.mark.parametrize( | |
| ("garbage"), | |
| [ | |
| b"", | |
| b"\x00\x00\x05\xa1" | |
| b"\x00\x00\x05\xa1\xff\xff" | |
| ], | |
| ) | |
| def test_decoder_clean_message(garbage: bytes): | |
| encoder = create_local_encoder(TEST_LOCAL_KEY) | |
| decoder = create_local_decoder(TEST_LOCAL_KEY) | |
| msg = RoborockMessage( | |
| protocol=RoborockMessageProtocol.RPC_REQUEST, | |
| payload=b"test_payload", | |
| version=b"1.0", | |
| seq=1, | |
| random=123, | |
| ) | |
| encoded = encoder(msg) | |
| decoded = decoder(garbage + encoded) | |
| assert len(decoded) == 1 | |
| assert decoded[0].payload == b"test_payload" |
allenporter
left a comment
There was a problem hiding this comment.
Please make the requested changes in a followup PR. I'd like to get this fix in.
So our expected behavior is that if it does not start with the verison bytes, it should start with 4 bytes that give the payload length.
That is not always happening - likely due to the buffer, but it's hard to say. This will fix the problem while we can explore more in depth about the buffer
Partly related to home-assistant/core#157599 ?
Talks about the decoding issue there