Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added phase 3 methods and test for remove bulk #91

Merged
merged 8 commits into from
Jul 26, 2017
34 changes: 33 additions & 1 deletion src/cache_manager/CacheManager.h
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,10 @@

#include <map>

#include "object_store/ObjectStore.h"
#include <vector>
#include <functional>
#include <algorithm>
#include "object_store/ObjectStore.h"
#include "cache_manager/EvictionPolicy.h"
#include "object_store/FullBladeObjectStore.h"
#include "common/Exception.h"
Expand All @@ -28,6 +28,8 @@ class CacheManager {
void put(ObjectID oid, T obj);
void prefetch(ObjectID oid);
void remove(ObjectID oid);
void removeBulk(ObjectID first, ObjectID last);
void prefetchBulk(ObjectID first, ObjectID last);

private:
void evict_vector(const std::vector<ObjectID>& to_remove);
Expand Down Expand Up @@ -184,6 +186,36 @@ void CacheManager<T>::remove(ObjectID oid) {
policy->remove(oid);
}

/**
* Prefetches a range of objects.
* @param first the first ObjectID to prefetch
* @param last the last ObjectID to prefetch
*/
template<class T>
void CacheManager<T>::prefetchBulk(ObjectID first, ObjectID last) {
if (first > last) {
throw cirrus::Exception("Last ObjectID to prefetch must be leq first.");
}
for (int oid = first; oid <= last; oid++) {
prefetch(oid);
}
}

/**
* Removes a range of objects from the cache.
* @param first the first continuous ObjectID to be removed from the cache
* @param last the last ObjectID to be removed
*/
template<class T>
void CacheManager<T>::removeBulk(ObjectID first, ObjectID last) {
if (first > last) {
throw cirrus::Exception("Last ObjectID to remove must be leq first.");
}
for (int oid = first; oid <= last; oid++) {
remove(oid);
}
}

/**
* Removes the cache entry corresponding to oid from the cache.
* Throws an error if it is not present.
Expand Down
17 changes: 16 additions & 1 deletion src/object_store/FullBladeObjectStore.h
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ class FullBladeObjectStoreTempl : public ObjectStore<T> {
const ObjectID& id) override;
typename ObjectStore<T>::ObjectStorePutFuture put_async(const ObjectID& id,
const T& obj) override;

void removeBulk(ObjectID first, ObjectID last) override;
void printStats() const noexcept override;

private:
Expand Down Expand Up @@ -208,6 +208,21 @@ bool FullBladeObjectStoreTempl<T>::remove(ObjectID id) {
return client->remove(id);
}

/**
* Removes a range of items from the store.
* @param first the first in a range of continuous ObjectIDs to be removed
* @param last the last in a range of continuous ObjectIDs to be removed
*/
template<class T>
void FullBladeObjectStoreTempl<T>::removeBulk(ObjectID first, ObjectID last) {
if (first > last) {
throw cirrus::Exception("First ObjectID to remove must be leq last.");
}
for (int oid = first; oid <= last; oid++) {
client->remove(oid);
}
}

template<class T>
void FullBladeObjectStoreTempl<T>::printStats() const noexcept {
}
Expand Down
7 changes: 7 additions & 0 deletions src/object_store/ObjectStore.h
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,8 @@ class ObjectStore {

virtual ObjectStoreGetFuture get_async(const ObjectID& id);
virtual ObjectStorePutFuture put_async(const ObjectID& id, const T& obj);

virtual void removeBulk(ObjectID first, ObjectID last);
};

template<class T>
Expand All @@ -106,6 +108,11 @@ typename ObjectStore<T>::ObjectStorePutFuture ObjectStore<T>::put_async(
return ObjectStorePutFuture();
}

template<class T>
void ObjectStore<T>::removeBulk(ObjectID first, ObjectID last) {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should there be comments indicating that these are placeholder methods?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why isn't this declared as a virtual method?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should I have declared it virtual void etc. here as it was declared as such above? Or are you asking why I even have this section of code at all?

If the latter, I included it because there are two store classes (Fullcache store and RDMAObjectStore) that I feared would not compile as they would not have the removeBulk method implemented.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it should be declared as a virtual method. The store classes can define that method -- and throw "not implemented" exception.

Copy link
Collaborator Author

@TylerADavis TylerADavis Jul 18, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, I'll take that approach in the phase two pull request as well, and also apply it to get_async() and put_async().

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realized that the other two store classes have actually not been getting compiled as there is no code that relies on them. As such, they are not up to date with the current interface. I think it may be best to simply open an issue to bring them up to date as opposed to fixing them here and in #81 . Do you agree?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've opened an issue for this in #95

return;
}

/**
* Constructor for an ObjectStoreputFuture.
* @param client_future the client_future that will be used for all
Expand Down
36 changes: 30 additions & 6 deletions tests/object_store/test_fullblade_store.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
#include "common/Exception.h"
#include "utils/Time.h"
#include "utils/Stats.h"
#include "client/RDMAClient.h"
#include "client/TCPClient.h"

// TODO(Tyler): Remove hardcoded IP and PORT
static const uint64_t GB = (1024*1024*1024);
Expand All @@ -27,7 +27,7 @@ static const uint32_t SIZE = 1;
* works properly.
*/
void test_sync() {
cirrus::RDMAClient client;
cirrus::TCPClient client;
cirrus::ostore::FullBladeObjectStoreTempl<cirrus::Dummy<SIZE>> store(IP,
PORT,
&client,
Expand All @@ -54,7 +54,7 @@ void test_sync() {
* Also record the latencies distributions
*/
void test_sync(int N) {
cirrus::RDMAClient client;
cirrus::TCPClient client;
cirrus::ostore::FullBladeObjectStoreTempl<cirrus::Dummy<SIZE>> store(IP,
PORT,
&client,
Expand Down Expand Up @@ -96,7 +96,7 @@ void test_sync(int N) {
* get an ID that has never been put. Should throw a cirrus::NoSuchIDException.
*/
void test_nonexistent_get() {
cirrus::RDMAClient client;
cirrus::TCPClient client;
cirrus::ostore::FullBladeObjectStoreTempl<int> store(IP, PORT, &client,
cirrus::serializer_simple<int>,
cirrus::deserializer_simple<int, sizeof(int)>);
Expand All @@ -113,7 +113,7 @@ void test_nonexistent_get() {
* Tests a simple asynchronous put and get.
*/
void test_async() {
cirrus::RDMAClient client;
cirrus::TCPClient client;
cirrus::ostore::FullBladeObjectStoreTempl<int> store(IP, PORT, &client,
cirrus::serializer_simple<int>,
cirrus::deserializer_simple<int, sizeof(int)>);
Expand All @@ -136,7 +136,7 @@ void test_async() {
* @param N the number of puts and gets to perform.
*/
void test_async_N(int N) {
cirrus::RDMAClient client;
cirrus::TCPClient client;
cirrus::ostore::FullBladeObjectStoreTempl<int> store(IP, PORT, &client,
cirrus::serializer_simple<int>,
cirrus::deserializer_simple<int, sizeof(int)>);
Expand Down Expand Up @@ -167,6 +167,22 @@ void test_async_N(int N) {
}
}

/**
* Tests that remove_bulk correctly removes items from the store.
*/
void test_remove_bulk() {
cirrus::TCPClient client;
cirrus::ostore::FullBladeObjectStoreTempl<int> store(IP, PORT, &client,
cirrus::serializer_simple<int>,
cirrus::deserializer_simple<int, sizeof(int)>);
for (int i = 0; i < 10; i++) {
store.put(i, i);
}

store.removeBulk(0, 9);
// Should fail
store.get(9);
}
auto main() -> int {
std::cout << "Starting synchronous tests." << std::endl;
test_sync(10);
Expand All @@ -183,5 +199,13 @@ auto main() -> int {
} catch (const cirrus::NoSuchIDException& e) {
}

try {
test_remove_bulk();
std::cout << "Exception not thrown in remove bulk test." << std::endl;
return -1;
} catch (const cirrus::NoSuchIDException& e) {
}

std::cout << "Test successful." << std::endl;
return 0;
}