How to use an ML tier-2 cache admission policy?

[pbhandar2]

Hello,

I am working on tier-2 cache admission for my research. I find that currently you can use the following types of tier-2
cache admission.

1. random: this uses a static probability to randomly pick an item to admit
2. dynamic-random: you can set a target write rate or a max device write rate and admission rate changes to meet these requirements
3. reject first: reject admitting the item the first time it is seen, admit it the next time

I saw that tiny-LFU which is a useful cache admission policy is not usable for the NVM cache. Is it because it does not
evict page every time it admits the page and that it does replacement at a region granularity rather than the page granularity?

I want to evaluate different workloads using the admission policies that CacheLib currently supports and implement
a ML cache admission policy that we compare against the baseline (random, dynamic-random, reject-first).

I wanted to ask if there are resources that anybody can point me to regarding how to integrate ML models into
CacheLib to use as a tier-2 admission policy.

[haowu14]

Before going into tiny LFU, I want to clarify the admission policies you mentioned above.
There are two places where you see the words of admission policy in cachelib: 1) allocator/NvmAdmissionPolicy.h; 2) navy/admission_policy/AdmissionPolicy.h.
They both reject items, but the intention for 2) was designed as a throttler that acts upon item size and device endurance. And you can see we didn't make it extendable, you can only pick RandomPolicy, assuming your workload is stable, or DynamicRandomAP where you only know the device endurance limit.

1. was designed to host heuristics, such as access frequency information, etc. So if you want to experiment with heuristics ideas, you can extend 1) and if your heuristics is already taking size and device endurance into account, you can make 2) admit all or just put DynamicRandomAP there and it will never throttle anything.

As for tiny-LFU:
What you mentioned is part of the reason. Also there is small item cache where we have many hash buckets with their own queue.

To use ML, you can extend allocator/NvmAdmissionPolicy.h and override the accept function. In the accept function, you can extract feature from the item, call the ML model to produce a binary decision of admit/reject. We don't have code to share since the ML model is proprietary but you can let me know if you have any questions.