Memory registered for RDMA never deregistered

[TylerADavis]

Currently, memory is registered for RDMA in two locations.The first of these is the send/receive buffers within the connection context, and the other is that whenever a message is sent or received, the serialized message/buffer to receive is registered. While the buffers for the connection context are deregistered, the other buffers are never deregistered.

There may be some performance impact if we have deregistration in the critical path.

One approach may be to have the RDMAMem struct automatically deregister its memory on destruction if the memory was registered.

[TylerADavis]

Currently, there is a clear method within RDMAMem that deregisters the memory, but it is never used.

According to the documentation for ibv_dereg_mr, the memory should be deregistered before it is freed.

For read operations, the RDMAMem is destroyed when apply() is called on the RDMAOpInfo, and the memory is automatically freed when the ClientFuture is destroyed.

For write operations, the memory is currently freed in on_completion, and it does not seem that the RDMAMem is ever destroyed.