Cannot read memory asynchronously, between processes

[funatsufumiya]

I tested with different processes, but read data won't change. Maybe while other process is writing something. I think some kind of mutex or double buffer is required here.

[kyr0]

Hmm, yeah, there are two cases here:

a) the change flag is changed by two writer processes. Because this acts as a toggle, it can create a situation where reader processes don't recognize changes. The data might change but the code doesn't "identify" the change. This could be solved by replacing the bit with a CRC checksum and checking the checksum for change.

b) two writer processes are writing and overwriting themselves. If the data is "the same", and readers "lack behind" in timing, they wouldn't see the change that happened "in between". Adding a revision number (a number that starts at 0 and increments with every change) would help here to identify "missed changes" on the reader side. If the revision counter increased > +1 for a reader (between the last read and the current read), it's clear that changes are missing. The complicated part here is that also he writer process needs to read the revision number before incrementing. There could still be a RACE condition where two writer processes read the revision number at the same time and increment by +1, overwriting their data. That's why this impl. would also need a lock. To come up with a solid implementation, every writer needs to randomly decide on a process id that with a very high chance is not taken by another writer process. The intuitive choice would be MD5'ing a UUIDv4. The alternative is to synchronize "process IDs" using a temporary file. This would be simpler as process IDs could just be numbers then... The "process id" would be written to the meta data in a lock field. If a writer process reads the revision number and the process id in the lock is not it's own, it needs to wait and re-read it unitil the lock is 0. If the lock is 0, the first writer process that writes it's own process id in the lock field, wins the RACE; reads the then-current revision number, writes its data, increases the revision number, sets the lock to 0 again and so on... to not overflow the revision number, max. 255 revisions would be supported if the revision field has 1 byte length.

However even this wouldn't solve the "What's the data that I've missed?" Question. To implement this, the library could allocate n more shared memory buffers (configurable to max 255 "backlog"); these would start with the index of the counter number (index), followed by the data type (see readme) and then the data. If the library detects that the counter has "jumped" more than +1 change, it could find the right data in those buffers, by checking the buffers, looking for the n+1 index (aka the data for the revision number indicated).

I don't have much time these days, but I'd welcome PR's if you like to play with this maybe?

[kyr0]

@funatsufumiya Had a few more thoughts... RACE was still possible with the previous idea; I edited my comment :)

[funatsufumiya]

@kyr0
Thank you for your prompt reply!

I was trying to use it like the so-called PUSH/PULL or PUB/SUB, but I was struggling because the data was only loaded correctly the first time.

The first thing I thought of was to create a couple of shared memory locations with similar names and swap them with a separate area that points to which one is being used, but I will try the method you adviced me :)

[funatsufumiya]

I also had an idea to create an area that corresponds to a mutex lock, and both the reader and writer use lock there, but it seemed to cause problems such as the locks not being removed permanently, so I would like to try various methods.

[funatsufumiya]

I just came up with the idea for PUSH/PULL style, that the shared memory should have a different (incremental) name each time and the reader should delete the memory once it has been read. That way, the write side would always write to a new area and not have to know the behavior of the read side, which may be the best way for my current needs. (Shared memory that points to the latest address may be necessary.)

[kyr0]

Yep, that would resemble parts of the idea of process_ids basically; you can also simply create a temporary file where the process ids of writers are named in.

SharedMemoryWriteStream write$ {/*name*/ "jsonPipe_" + writerProcessId, /*size*/ 65535, /*persistent*/ true};

Readers need to watch the shmWriterProcessList file in read mode to identify new possible shared memory sources dynamically. Writers would prefix the name of the shared memory with their process id. Readers do the same to watch and read all shared memory locations available. Now, you would not know whats the most recent data.

To identify that, you can simply prefix your actual data with a few bytes (as a prefix) encoding a current timestamp (e.g. ms precision UNIX timestamp) when data is written. The reader then uses this timestamp info to select the most recent one given the current system clock time.

      // or whatever datatype you're using
      float* dataReadPtr = read$.readFloatArray();
      dataReadPtr[0] // timestamp
      dataReadPtr[...] // actual data

This way you'd basically build a library with a protocol on top of libsharedmemory. You could release it as a separate header-only library :)) I'm sure people would be happy. I'd link your project in case you'd release it.

[funatsufumiya]

@kyr0 Thank you! I'll try them!

[funatsufumiya]

@kyr0
Upon further investigation, I found that there are already several high-level IPC Messaging implementations I am looking for, and here are links to representative ones.

• shadesmar
• alephzero
• iceoryx

(And also, but much higher level, may you already know)

• nanomsg
• NNG
• ZeroMQ

Referenced Issue:
Squadrick/shadesmar#56

[funatsufumiya]

By the way, if we look at it not as a messaging protocol but in the direction of Ephemeral DB, I think SimDB (and many more) would be the category of high-level implementations.

[funatsufumiya]

I couldn't find this before, but cpp-ipc seems to be an IPC implementation with a simple mechanism like PUBSUB. (The number of receivers seems to be limited to 32.)

https://github.com/mutouyun/cpp-ipc

(PS: It was not a cross-platform implementation...)

[funatsufumiya]

Looking at cpp-ipc, I'm starting to wonder if libsharedmemory itself could have a simple channel mechanism (otherwise anyone would be reinventing the wheel as a result), so I'll re-open the Issue itself in the meantime and see if a PR can be created.

[kyr0]

Oh thank you for all the research you've done. I think we could add those findings to the README.md as well. A section line "Alternatives" could be helpful for others I guess?

And yeah, there's a lot of alternatives out there already -- I personally think that simplicity is key for good software quality. The most simple solution that does the job is often the best IMHO.. it allows code to be understood by anyone, adapted, maintained, fixed etc. pp. That's why I sometimes reinvent the wheel on purpose.. just to get to a solution that is waaay simpler than the madness of overcomplexity that we can often find out there :)

[funatsufumiya]

I generally agree, but in this case, many use cases need asynchronous connection from multiple applications, so I wonder that some simple asynchronous support would be good.