limit number of open files

[mockersf]

Objective

• Limit the number of open files
• fix: limit the number of concurrent load_internal calls to avoid exce… #17377, but scoped to the FileAssetReader,
• Fixes On iOS, loading many assets in AssetMode::Processed causes some assets to fail to load #14542
• Running the bistro scene on macOS gives errors in the log, and some texture are randomly not loaded:

ERROR bevy_asset::server: Encountered an I/O error while loading asset: Too many open files (os error 24)

Solution

• on iOS and macOS, add a Semaphore to ensure we're not opening more than 128 files in parallel

Testing

• cargo run --release --package bistro works without errors in the log

[andriyDev]

I'm a little nervous to do this: you could end up with every running loader acquiring a single file, but then needing a second file to complete, and then being deadlocked.

It's unclear whether this is better than just failing to load. I guess so since it's less likely that users have 128 assets that also need a second file load to complete (e.g., an immediate load). Though having 128 gltfs that reference other textures isn't completely impossible.

[andriyDev]

Also this doesn't take into consideration multiple asset sources. This might need to be a global semaphore?

[mockersf]

I'm a little nervous to do this: you could end up with every running loader acquiring a single file, but then needing a second file to complete, and then being deadlocked.

I've added a timeout of 500ms on acquiring the semaphore, and it continues like before if not possible in that timeout, so it will never block

[mockersf]

Also this doesn't take into consideration multiple asset sources. This might need to be a global semaphore?

I would prefer to have an actual case where this happens before adding it. I'm not a fan of global things, and apple OS + multiple asset sources + many simultaneously open files starts becoming a very specific use case

[andriyDev]

I don't think we should be treating multiple asset sources as some exotic use case. Users often want to store data in different locations.

But whatever, I'm not gonna block on it. Could we at least add a TODO to consider making the semaphore global to work across multiple asset sources?

[mockersf]

now global

[andriyDev]

Windows doesn't have a limit though. I'd prefer to avoid it for windows? Wdyt?

[mockersf]

hoo so it's windows that doesn't have a limit!

[andriyDev]

Where did 512 come from?

[mockersf]

on macOS it works best with the limit (256) / 2 => 128
so I did the same of linux: 1024 / 2 => 512

Added a comment about it