Fix behavior of SilkMarshal.StringToPtr and related methods on Linux

[Exanite]

Summary of the PR

This fixes #2374 by checking for whether the platform is Windows and using 2-byte LPWStrs if it is Windows and 4-byte LPWStrs if it is not.

Related issues, Discord discussions, or proposals

#2374

Further Comments

This pull request includes the changes made in #2376.
In that pull request, I added a test project (Silk.NET.Core.Tests) which is used here to test the changes I made.

[Exanite]

I built the package locally and overrode the package version that I was using.

Doing this makes my project run successfully on both Windows and Linux without any further changes.
I also ran the test cases I wrote on both Windows and Linux. The CI Build seems to take care of Mac.

[Perksey]

Just a few comments, looks great!

[Exanite]

I changed the code to use Encoding.UTF32 for both converting to a pointer and from a pointer. The test cases pass, but I'm still wondering if this is correct (see my reply to your review comment).

I also don't test for any edge cases. I'm considering about including some of those.

[Perksey]

Will merge following #2376

[Exanite]

Hold on. I want to do a bit more research and make sure the behavior here is correct.

[Exanite]

Interesting. It looks like on Windows, LPWStr is UTF-16 as expected. However, on Linux, LPWStr is UTF-32.

For example, the memory for 🧵:

• On Linux: 0xf5f901
• On Windows: 0x3ed8f5dd

I checked this by inspecting the memory of L"🧵" on both Windows and Linux.

The codepoint for 🧵 is U+1F9F5.
The surrogate pair for 🧵 is 0xd8e3 and 0xddf5.

Ignoring byte order, everything matches up.

This means that it is indeed correct to use UTF32 on non-Windows platforms.
I'll update the code to reflect this.

[Exanite]

I found another issue where on Windows, characters like emojis get garbled.
This apparently was always the case.

I'll look into fixing this as well.

[Exanite]

Okay, never mind. That technically is expected behavior since that case is testing LPStr (which is 8 bits, but not necessarily UTF-8) and non-ascii characters are not guaranteed to be supported.