Skip to content

Editing and saving UTF-16 LE BOM files with certain symbols within them mangles file contents. #752

@Nereithr

Description

@Nereithr

Synopsis / Background info

Hello! Ran into this after running New-ModuleManifest, which creates a UTF16 LE BOM file. Uncommented a key, edited the value, saved , tried to import the data file and it failed to parse. I look back at the file and half of it is missing.

Issue

The issue manifests as file contents disappearing and then potentially reappearing at other parts of the document. It occurs after an existing hash symbol (#) is removed from one of the lines, after which the file is irrevocably damaged and other glitches occur (such as contents reappearing when writing new text).

Steps to reproduce

  1. Open a terminal
  2. New-ModuleManifest SomeModule.psd1
  3. edit SomeModule.psd1
  4. Remove some of the hash symbols. Removing the first hash symbol occuring in the file triggers it consistently
  5. Save the file
  6. Reopen the file in edit or any other editor to confirm the issue

The issue appears to occur specifically when hash symbols (#) are present in a UTF-16 LE BOM file. It may also have to do with the other symbols present in a psd1 file (@ {} ()). I could not consistently reproduce it outside the general structure of a psd1 file. I could reproduce it while manually copying over a few hashtables and comments to a new file. It's also not a quirk of New-ModuleManifest generation as the cmdlet produces no hidden symbols besides line endings and I could reproduce it on a manually created PSD1 structure.

The issue does not occur when the file is converted to UTF-8, including converting it to UTF-8 using Edit itself. You can edit and save it just fine.

The issue does not occur in other editors tested (VS Code, Notepad++, micro).

Apologies in advance if I missed an existing issue, I searched for UTF-16 / encoding / psd1 and found nothing besides the SSH issues, which may or may not be related.

Environment

  • Windows 11 Pro 25H2
  • Edit 1.2.1

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions