Added duplicate key detection forked_projects_util.py

[Kcruz28]

This PR adds a check for duplicate keys in forked_projects.json. I added a new function, find_duplicate_json_keys, which scans the raw JSON file to detect duplicate keys before it is parsed. I then updated run_checks_sort_fp to report an error if any duplicates are found. This helps catch cases where duplicate keys would otherwise be silently overwritten, making the checks more reliable.

Example error:
ERROR: On file format_checker/forked-projects.json: Duplicate in forked-projects.json keys detected: ['https://github.com/apache/incubator-shardingsphere']

[darko-marinov]

Good starting point but some changes are needed to accept this in the checker.

[darko-marinov]

Use Json library to read the file. It's presumably already done somewhere in this script.

[Kcruz28]

json.load() is the standard way to parse JSON, but it automatically collapses duplicate keys by keeping only the last occurrence. Because of this behavior, any duplicate keys in the original file are lost during parsing. For that reason I did not use json.load().

[darko-marinov]

Thanks for clarifying about json.load(); I wasnt' aware of the default behavior. It seems that json.loads(..., object_pairs_hook=...) allows for checking for duplicates (and doesn't require you to parse keys, assuming all key-value pairs are in the same line).

[darko-marinov]

Does the sorting check strictly increasing or just non-decreasing? Can we make it strictly increasing so that it subsumes the check for duplicates?

[Kcruz28]

Making the sorting check strictly increasing would catch duplicates in the loaded data. But json.load() collapses duplicates when parsing, so any duplicates in the original file wouldn’t be detected.