Drop BoltDB from write-cache #3091

End-rey · 2025-01-27T13:06:48Z

Closes #3076.

Do I understand correctly that the workers were only for parallel processing from the BoltDB?

codecov · 2025-01-27T13:09:05Z

Codecov Report

Attention: Patch coverage is 33.61345% with 79 lines in your changes missing coverage. Please review.

Project coverage is 22.23%. Comparing base (25314f5) to head (3364ab5).
Report is 12 commits behind head on master.

Files with missing lines	Patch %	Lines
pkg/local_object_storage/writecache/writecache.go	40.25%	45 Missing and 1 partial ⚠️
cmd/neofs-lens/internal/writecache/get.go	0.00%	9 Missing ⚠️
cmd/neofs-lens/internal/writecache/root.go	0.00%	6 Missing ⚠️
pkg/local_object_storage/writecache/flush.go	33.33%	6 Missing ⚠️
cmd/neofs-lens/internal/writecache/list.go	0.00%	4 Missing ⚠️
pkg/local_object_storage/writecache/storage.go	0.00%	4 Missing ⚠️
cmd/neofs-lens/internal/printers.go	0.00%	1 Missing ⚠️
pkg/local_object_storage/writecache/delete.go	0.00%	1 Missing ⚠️
pkg/local_object_storage/writecache/status.go	0.00%	1 Missing ⚠️
pkg/services/control/server/object_status.go	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3091      +/-   ##
==========================================
- Coverage   22.40%   22.23%   -0.18%     
==========================================
  Files         763      762       -1     
  Lines       58632    58276     -356     
==========================================
- Hits        13136    12956     -180     
+ Misses      44603    44436     -167     
+ Partials      893      884       -9

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

pkg/local_object_storage/writecache/flush.go

CHANGELOG.md

cmd/neofs-lens/internal/writecache/get.go

pkg/local_object_storage/shard/shutdown_test.go

pkg/local_object_storage/writecache/flush.go

pkg/local_object_storage/writecache/writecache.go

End-rey · 2025-01-28T16:53:18Z

An error is currently being returned after migration. Can you check which errors may appear, specifically I am concerned when there are no default bucket or other errors inside the database View.

pkg/local_object_storage/writecache/writecache.go

roman-khimov · 2025-01-29T08:27:34Z

pkg/local_object_storage/writecache/flush.go

 	// defaultFlushInterval is default time interval between successive flushes.
-	defaultFlushInterval = time.Second
+	defaultFlushInterval = 10 * time.Second


This change should be reflected in documentation.

I did't find where in the documentation it is specified about the flush interval for the write-cache. It's not set up anywhere.

pkg/local_object_storage/shard/shutdown_test.go

pkg/local_object_storage/writecache/writecache.go

carpawell

Do I understand correctly that the workers were only for parallel processing from the BoltDB?

TBH, I do not understand what is the key difference b/w db and FSTree if we are talking about flushing. I think it is possible to speed up flushing if there are 2+ workers. We do spend some time on object reading from WC cache and its unmarshaling. @roman-khimov

Also, we store raw objects on FS now, I think closing this issue should be done with more precise WC size handling:

neofs-node/pkg/local_object_storage/writecache/state.go

Lines 11 to 13 in 9bc61c5

    
           func (c *cache) estimateCacheSize() uint64 { 
        
           	return c.objCounters.DB()*c.smallObjectSize + c.objCounters.FS()*c.maxObjectSize 
        
           }

@roman-khimov

carpawell · 2025-01-29T11:47:11Z

pkg/local_object_storage/writecache/delete.go

-		storagelog.Write(c.log,
-			storagelog.AddressField(saddr),
-			storagelog.StorageTypeField(wcStorageType),
-			storagelog.OpField("db DELETE"),


we also can drop "fstree" in every WC operation log since it is now clear that it was dropped from FSTREE

carpawell · 2025-01-29T11:48:12Z

pkg/local_object_storage/writecache/doc.go

-// Write-cache has 2 components:
-// 1. Key-value (bbolt) database for storing small objects.
-// 2. Filesystem tree for storing big objects.
+// Write-cache uses filesystem tree for storing objects.


"file system" (separate)?

carpawell · 2025-01-29T12:23:26Z

pkg/local_object_storage/writecache/writecache.go

+	c.modeMtx.Unlock()
+
+	// Migration part
+	if !readOnly {


can we prevent ! and just invert branches?

carpawell · 2025-01-29T12:27:39Z

pkg/local_object_storage/writecache/writecache.go

+	return nil
+}
+
+func (c *cache) migrate() error {


can we have some simple migrate test?

carpawell · 2025-01-29T12:30:09Z

CHANGELOG.md


 ### Updated

 ### Updating from v0.44.2
+`small_object_size`, `workers_number`, `max_batch_size` and `max_batch_delay`
+paramteters are removed from `writecache` config. These parameters are related
+to the BoltDB part of the write-cache, which is dropped from the code.


add, please, a few words that migration will be performed automatically

carpawell · 2025-01-29T12:35:48Z

cmd/neofs-lens/internal/writecache/root.go

-	db, err := writecache.OpenDB(vPath, true)
+// openWC opens and returns read-only writecache.Cache located in vPath.
+func openWC() (writecache.Cache, error) {
+	ws := writecache.New(writecache.WithPath(vPath))


Suggested change

ws := writecache.New(writecache.WithPath(vPath))

wc := writecache.New(writecache.WithPath(vPath))

carpawell · 2025-01-29T12:44:07Z

pkg/local_object_storage/writecache/options.go

-		}
-	}
-}
-
 // WithNoSync sets an option to allow returning to caller on PUT before write is persisted.
 // Note, that we use this flag for FSTree only and DO NOT use it for a bolt DB because


this line should be dropped

roman-khimov · 2025-01-29T12:58:46Z

Estimations and workers are separate things to be done in separate issues, please open them if they're not here yet. Eventually I'd like wc to keep it's object list in memory (with sizes) and yes, it needs better flushing mechanism. But this doesn't change anything anything here, Bolt is to be removed and it is removed.

Although there is `ignoreErrors` flag, but when `flushObjects` occurs in FSTree, the error should be returned as with Bolt. Signed-off-by: Andrey Butusov <[email protected]>

Previously, the write-cache has 2 components: bolt database and fstree. BoltDB is absolutely useless for write caching given that the cache uses SSD, so it was decided to drop this database. Drop all code related to BoltDB. Migrate from this database to the main storage, by flushing objects. Remove config parameters for the write-cache: `small_object_size`, `workers_number`, `max_batch_delay`, `max_batch_size`. Update docs. Rewrite lens commands for the current version of the write-cache. Closes #3076. Signed-off-by: Andrey Butusov <[email protected]>

carpawell · 2025-01-29T15:02:02Z

Estimations and workers are separate things to be done in separate issues, please open them if they're not here yet.

Do not agree. It used to be parallel, it is not now, it is a degradation. It used to have incorrect estimations only because of the existing bbolt base, now we do not have it but still, our expectation about WC size is numOfObjs*theBiggestPossibleObj. I see no better place for these fixes except this PR.

Opened: #3100, #3101.

roman-khimov · 2025-01-29T16:37:18Z

It used to be parallel

Bolt-only. FSTree is all the same.

End-rey self-assigned this Jan 27, 2025

End-rey requested review from roman-khimov, carpawell and cthulhu-rider as code owners January 27, 2025 13:06

carpawell reviewed Jan 27, 2025

View reviewed changes

pkg/local_object_storage/writecache/flush.go Show resolved Hide resolved

End-rey force-pushed the 3076-drop-boltdb-from-write-cache branch from 22d9414 to 9957225 Compare January 28, 2025 12:24

roman-khimov reviewed Jan 28, 2025

View reviewed changes

End-rey force-pushed the 3076-drop-boltdb-from-write-cache branch from 9957225 to b9784dd Compare January 28, 2025 16:48

roman-khimov reviewed Jan 29, 2025

View reviewed changes

carpawell reviewed Jan 29, 2025

View reviewed changes

End-rey added 2 commits January 29, 2025 17:32

writecache: fix test flush on moving to degraded mode

b4e5d1b

Although there is `ignoreErrors` flag, but when `flushObjects` occurs in FSTree, the error should be returned as with Bolt. Signed-off-by: Andrey Butusov <[email protected]>

End-rey force-pushed the 3076-drop-boltdb-from-write-cache branch from b9784dd to 3364ab5 Compare January 29, 2025 14:33

This was referenced Jan 29, 2025

ignoreErrors in WC, dump shard routine, etc #3099

Open

Parallelize WC flush process #3100

Open

End-rey requested review from carpawell and roman-khimov January 29, 2025 14:55

carpawell mentioned this pull request Jan 29, 2025

Fix WC size estimations #3101

Closed

carpawell approved these changes Jan 29, 2025

View reviewed changes

roman-khimov approved these changes Jan 30, 2025

View reviewed changes

roman-khimov merged commit 36f1d3e into master Jan 30, 2025
22 checks passed

roman-khimov deleted the 3076-drop-boltdb-from-write-cache branch January 30, 2025 19:25

End-rey mentioned this pull request Jan 31, 2025

Change WC size estimations #3106

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Drop BoltDB from write-cache #3091

Drop BoltDB from write-cache #3091

End-rey commented Jan 27, 2025

codecov bot commented Jan 27, 2025 •

edited

Loading

End-rey commented Jan 28, 2025

roman-khimov Jan 29, 2025

End-rey Jan 29, 2025

carpawell left a comment •

edited

Loading

carpawell Jan 29, 2025

carpawell Jan 29, 2025

carpawell Jan 29, 2025

carpawell Jan 29, 2025

carpawell Jan 29, 2025

carpawell Jan 29, 2025

carpawell Jan 29, 2025

roman-khimov commented Jan 29, 2025

carpawell commented Jan 29, 2025

roman-khimov commented Jan 29, 2025

	func (c *cache) estimateCacheSize() uint64 {
	return c.objCounters.DB()c.smallObjectSize + c.objCounters.FS()c.maxObjectSize
	}

	ws := writecache.New(writecache.WithPath(vPath))
	wc := writecache.New(writecache.WithPath(vPath))

Drop BoltDB from write-cache #3091

Drop BoltDB from write-cache #3091

Conversation

End-rey commented Jan 27, 2025

codecov bot commented Jan 27, 2025 • edited Loading

Codecov Report

End-rey commented Jan 28, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

carpawell left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

roman-khimov commented Jan 29, 2025

carpawell commented Jan 29, 2025

roman-khimov commented Jan 29, 2025

codecov bot commented Jan 27, 2025 •

edited

Loading

carpawell left a comment •

edited

Loading