Multithreaded Cache Manager Use

[TylerADavis]

It is feasible that an application could have multiple threads working concurrently using the same cache. There should be a test to ensure that the cache manager is thread safe and behaves properly in these situations.

[TylerADavis]

@jcarreira having a thread safe cache manager means that the eviction policy needs to be thread safe as well. Would it be better to force the user to enforce thread safety, or to simply put a lock around the policy that allows only one operation at a time?

[TylerADavis]

Currently the cache is based on an std::map<ObjectID, struct cache_entry>, it may be better to switch to a cuckoohashmap as it is inherently thread safe, vs the std::map where each access would have to be wrapped in a lock. What do you think?

[jcarreira]

Using a cuckoohashmap might not be a bad idea because it has better latency and multicore scalability than a regular std::map.

The only big downside seems to be the creation of an extra dependency.

[TylerADavis]

Currently the cuckoohashmap is used in the RDMA client to map ObjectIDs to BladeLocations, but we aren't making use of it anywhere else

[TylerADavis]

It seems like a good idea to put the cuckoohashmap into the cache manager. However, one challenge may be the way its find method returns values.

std::map::find(key) generally returns an iterator that then returns a reference to the object corresponding to a given key, while cuckoohashmap::find() will either return a copy of the object, or copy the object into a specified location. This issue also presents itself in the function cuckoohashmap::update( const K& key, V && val), which replaces the object stored under key with a copy of val.

The issue with update could be resolved by instead using cuckoohashmap::update_fn(), which can modify the found object in place. This would be useful for changing the value of cache_entry.cached or cache_entry.obj while minimizing copying.

However, the issue with find still remains, and would likely cause an extra copy to be incurred, which may be unacceptable.