feat: change all collections to ordered collections

[Michael-Wamae]

• change all collections to ordered collections
  Fixes Allow for deterministic sorting while serializing collections #2329

[sonarqubecloud]

Quality Gate failed

Failed conditions
66.4% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube Cloud

[darrelmiller]

@Michael-Wamae Sorry for the randomization, @baywet and I have had some intense negotiations and have come to a compromise.

Facts:

• Emitting collections in a non-deterministic order is a problem for customers that we have had multiple requests to fix.
• Swashbuckle has the ability for developers to control the ordering of collections
• ASPNET would like to sort collections alphabetically
• OrderedDictionary has performance cost and is not supported in downrev versions of .NET
• No customer has asked for us to maintain the order of the input document (Darrel does not count as a customer)
• Future versions of .NET will support using the collection initialization syntax for interfaces

Compromise:

• Go back to using Dictionary during reading to avoid perf costs and need to implement a custom implementation for downrev .net
• Wait for customer signal to implement maintaining of input order
• Change the API surface area back to IDictionary to allow future implementations to use OrderedDictionary without it being a breaking change and continue allowing Swashbuckle customers to provide custom ordering
• IDictionary will require customers to explicitly provide the type until .NET adds implicit support
• Add WriterSettings option to enable sorting using a comparer to meet the need for deterministic ordering

Sorry again for sending you down the wrong path. Hopefully, what we have learned here will reduce future maintenance work.

/cc @baywet

[bkoelman]

@darrelmiller Thanks for the update.

Just to clarify, does this mean all collection properties will be reverted back to interfaces (including lists/sets), or does it only apply to dictionaries?

Add WriterSettings option to enable sorting using a comparer to meet the need for deterministic ordering

A comparer can only be used if the comparison obeys transitivity, which isn't always the case (example here). Although I'm not really concerned about such a limitation.

If all collections are interfaces again, this enables us to preserve insertion order (which we need in some cases, such as the declaration order of API entity properties). Passing a custom comparer can't solve that. What's the plan here?

[baywet]

Closing as superseded by #2363
@bkoelman the revert applies only to the dictionaries for the time being, but I encourage you to follow up on the other PR.

A comparer can only be used if the comparison obeys transitivity

Can you expand on this comment please? what impacts are you anticipating on this lib.

[bkoelman]

the revert applies only to the dictionaries for the time being, but I encourage you to follow up on the other PR.

Thanks for the clarification. I'll take another look after the new PR is merged.

A comparer can only be used if the comparison obeys transitivity

Can you expand on this comment please? what impacts are you anticipating on this lib.

The blog post I linked to explains it. For example, consider a list with content ['B', 'a', 'b', 'A']. This can't be sorted using .NET APIs with a case-insensitive comparer, combined with the requirement that insertion order must be preserved when only the casing differs.

I don't have a concrete use case, but it's something to be aware of. Just assuming any sorting can be accomplished by taking a user-defined comparer simply isn't true. Using a comparer in .NET requires that a stable/complete/total ordering (I don't know the proper mathematical terminology) exists.

[bkoelman]

The takeaway is that by offering collection interfaces instead of concrete types, users can define their own collections to honor such properties, and do so in an efficient way.

[bkoelman]

Another case where this matters is culture-sensitive comparers. For example, in German, the letter ß, called Eszett, is typographically equivalent to ss. See https://en.wikipedia.org/wiki/%C3%9F.

[baywet]

Thank you for the additional information.

For that German letter, do you typically expect it to appear after "B" when sorted alphabetically, or after "S" ?

[bkoelman]

I have no idea. I'm not German. It's just an example to make my point.