inquire about fuse mirror stability/performance

[cdarau]

Environment

• Windows version: 10 Enterprise N
• Processor architecture: x64
• Dokany version: 1.0.3.1000
• Library type (Dokany/FUSE): FUSE

Check List

• I checked my issue doesn't exist yet
• My issue is valid with mirror default sample and not specific to my user-mode driver implementation
• I can always reproduce the issue with the provided description below.
• I have updated Dokany to the latest version and have reboot my computer after.
• I tested one of the last snapshot from appveyor CI

Description

The fuse mirror appears to have some stability issues. I followed the wiki documented steps for FUSE and I ended up with a mounted drive. But performing some of the usual filesystem actions on it was barely possible. For example, opening an image file or an audio one. Also, I got errors when trying to copy files from and to the mounted drive.

[Liryna]

Hi @cdarau ,

Thank you for the report.
I understand what you mean but I think here we should probably create different issues or maybe look at how to improve it.
Any analyze, contributions would be appreciated here !

[ebekker]

@Liryna -- are the stability/performance issues cited by @cdarau an artifact of the FUSE wrapper, or do you think they are pby tied deeper into the actual Dokan library?

[Liryna]

Hi @ebekker,

It is related to the FUSE wrapper. The mirror and Dokan library are fast and stable.
From my point of view, it should be reviewed and why not even rewrote. A good use of the dokan library and a conversion to FUSE wrapper calls would make it a great wrapper !

I am focusing to improve the dokan sys / library / mirror and try to keep the FUSE wrapper working but I am really looking for some helps on the FUSE wrapper because I am lacking time.

[ebekker]

This is not FUSE-specific (so maybe I should post this discussion in another ticket?) but can you indicate how you feel about Dokan's performance? I didn't see if there were any benchmarks tested or published in this repo, so I'm thinking about doing some tests, but wanted to make sure I wasn't already duplicating existing work.

What I'm looking to get out of this for myself, is to see if the performance of going through Dokan as a filesystem is comparable to going through the native filesystem direclty or what would the marginal increase in overhead (decrease in throughput) typically amount to? Obviously this is heavily dependent on what the underlying Dokan FS impl is doing, but assuming that is super fast and efficient, how performant is the "Dokan layer".

[Liryna]

Hi @ebekker ,

Dokan is fast. Doing a benchmark would be really linked to the hardware so I prefer to let people test themselves.
But if you compare with a native filesystem directly it is for sure you will be disappointed. NTFS do some really beautiful magic tricks and without needs to send requests to userland so the performance are ofc better.
Dokan is able to handle the load and needs of all projects generally.
Like MLVFS that handling video files of 30gb (http://www.magiclantern.fm/forum/index.php?topic=13152.0)

heavily dependent on what the underlying Dokan FS impl is doing

Yes, you are absolutely right ! If for example, your FS is doing some remote request or slow complex/slow mechanism to get the files, this will be the bottleneck for sure and by far.

After, like every project, Dokan can be improved. It is an old dead project that we chose to bring back with a friend some years ago and with the community 💘 we were able to do a LOT about features and stability.
We are still working to handle all the FS operations (IFSTest https://ci.appveyor.com/project/Maxhy/dokany/build/1.0.5-282/job/1fsh0m6i2muyvrgf#L422) and after will focus on dokan 2.0 #307 that will give a real boost to dokan !

I hope this answer your question and if you have ideas/contributions to improve dokan this would be 👍 !!!

[ebekker]

Thank you for the insight! I'm thinking about using Dokan to write a FS in Dokan that would ultimately map back to a regular file or set of files on a host's local disk but perform encryption.

I know there are some other projects that use Dokan for something along these lines already (e.g. Keybase) but these have typically been for "special-purpose" and light use, for example store a few hundred text files or documents that get used infrequently.

My goal is to provide a FS that can support heavy and sustained use, such as the backing store for a database server or application server. I've worked with commercial offerings in this space before and they typically incur a 10%-15% performance penalty over writing to the disk directly to account for the encryption and access-control checks.