How to adjust set rate in kvcache_l2_wc

[junwoo0127]

@dreiss @willbailey @javache
We are currently trying to test the kvcache workload by comparing the different set rate. And we have found out in the code we have to change the cachelib/allocator/nvmcache/NavyConfig.cpp file configMap["navyConfig::maxWriteRate"] = folly::tostd::string(160);

But after even changing this rate, there is no difference in the set rate. We need more details.
Thank you.

[daebang]

Hello @junwoo0127
Please provide the file name and function name as well. Also the command you used for the test.
They need to know the purpose of the modification and what's expected result and the actual result.

[junwoo0127]

Adjust code(prediction)

1. file name : cachelib/allocator/nvmcache/NavyConfig.cpp

2. function name : configMap["navyConfig::maxWriteRate"] = folly::tostd::string(160);

3. file name : ./opt/cachelib/test_configs/ssd_perf/kvcache_l2_wc/config.json

4. function name: set ratio

5. build cmd : ./contrib/build.sh

6. run cmd : ./opt/cachelib/bin/cachebench -json_test_config ./opt/cachelib/test_configs/ssd_perf/kvcache_l2_wc/config.json --progress_stats_file=test.log

[wonglkd]

Hi @junwoo0127 , I'm not from Meta, but since you mentioned kvcache_l2_wc, have you tried changing the set ratio?

CacheLib/cachelib/cachebench/test_configs/ssd_perf/kvcache_l2_wc/config.json

Lines 41 to 46 in 1f60086

	"loneGetRatio": 8.2e-06,
	"loneSetRatio": 0.21,
	"setRatio": 0.0,
	"popDistFile": "pop.json",
	"setRatio": 0.0,
	"valSizeDistFile": "sizes.json"

I am guessing this should be passed to here:

CacheLib/cachelib/cachebench/workload/WorkloadDistribution.h

Lines 38 to 41 in 1f60086

	opDist_({config_.setRatio, config_.getRatio, config_.delRatio,
	config_.addChainedRatio, config_.loneGetRatio,
	config_.loneSetRatio, config_.updateRatio,
	config_.couldExistRatio}),

I believe maxWriteRate is a parameter for the DynamicRandomAdmissionPolicy flash admission policy and would just cap the number of sets by dropping them randomly to achieve the maximum flash write rate.

You may also wish to look at this if you have not: https://cachelib.org/docs/Cache_Library_User_Guides/Developing_for_Cachebench/#simple-config-with-key-value-sizes-and-popularity-distribution

kvcache_l2_reg may also be useful, it already has sets.

Hope it helps!

[junwoo0127]

Thank you, Wong! I first lowered the setratio value to 0.1 and tested it, but there wasn't any noticeable change. Oddly enough, when I significantly reduced the getratio, there was a significant difference. However, I didn't see any effect when I adjusted the setratio. I'll try running a full test for five days with just the set ratio changed.

Additionally, I would like your advice and feedback of the two other parameters I am assuming that might have affect to the setrate.

• maxWriteRate (cachelib/allocator/nvmcache/NavyConfig.cpp file configMap)
  
  

• maxNumWrites (cachelib/allocator/nvmcache/NavyConfig.h2)
  
  

Thank you for sharing your precious idea!!

[wonglkd]

Hi @junwoo0127 , I don't think either of those parameters will affect the number of sets that are issued. The first would affect the rate of sets that are rejected, and the second is a concurrency setting (how many simultaneous sets). You will want to look at the workload generator in CacheBench that you are using.

I am not sure what you meant by lower (in the example you are using, setRatio is set to 0), so you'd want to increase it. In that json file, it is also set twice -- you'd want to remove the duplicate entry in the json file. You should also try the other benchmark kvcache_l2_reg as a sanity check.

You should be seeing a significant change immediately once you tweak setRatio.

Best regards,
Daniel

[junwoo0127]

I've meant by lowering the average set rate.(OCP spec 2.07)
Currently, after the kvcache full-test, the average set rate is 2600(way over the ocp spec).


We've thought that this might influence the latency of the Read/Write so were trying to adjust the set rate close to 1500/s.

The default setting of set ratio of config.json is 0.0.

There is no instructions of how much we should lower the rate. Do you have any recommendations?(ratio to set rate of 1500 like OCP spec)